Vertical government affairs big model service method and system based on interactive learning
By building a graphical knowledge base and utilizing pre-trained language models, combined with reinforcement learning to optimize the intelligent question-and-answer system, the problem that existing systems cannot handle complex natural language queries is solved, personalized and context-related answers are achieved, and intelligent question-and-answer system is improved Quality and efficiency.
Patent Information
- Application Number
- CN202411304227.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-09-19
AI Technical Summary
Existing intelligent question-and-answer systems are difficult to handle complex natural language queries, lack personalized and context-related answers, and interactive learning methods fail to effectively utilize domain user feedback information.
The vertical government affairs big model service method based on interactive learning is adopted, and a graphical knowledge base is built by obtaining government affairs data and user query logs, and the pre-trained language model and reinforcement learning optimization model are used to optimize the model with user feedback information.
It realizes more accurate, personalized and context-related natural language replies, improves the quality and efficiency of intelligent Q&A, and dynamically adapts to user needs.
Smart Images

Figure CN118820448B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to big model technology, and in particular to a vertical government affairs big model service method and system based on interactive learning. Background Art
[0002] In the existing technology, intelligent question answering systems usually adopt rule-based or retrieval methods, which are difficult to handle complex natural language queries and cannot provide personalized and context-related answers. These systems rely on predefined rules or knowledge bases, lack flexibility and scalability, and are difficult to adapt to changing user needs and domain knowledge.
[0003] In order to improve the performance and user experience of intelligent question-answering systems, some studies have tried to introduce user feedback and interactive information and optimize question-answering models through feedback-driven learning methods. However, existing interactive learning methods mainly target general fields and lack consideration of the characteristics of vertical fields, making it difficult to effectively utilize feedback information from users in the field. At the same time, existing methods usually use simple feedback forms, such as clicks and ratings, which cannot capture users' fine-grained preferences and intentions, limiting the effectiveness of interactive learning. Summary of the invention
[0004] The embodiments of the present invention provide a vertical government affairs big model service method and system based on interactive learning, which can solve the problems in the prior art.
[0005] According to a first aspect of the embodiments of the present invention,
[0006] Provides vertical government affairs big model service methods based on interactive learning, including:
[0007] Obtain government data and user query log data for each vertical category, clean, annotate and structure the government data, build an ontology knowledge base for the vertical category, and based on the ontology knowledge base, capture the semantic associations between multiple types of nodes and multiple relationship edges by combining multiple relationship paths and the attention node aggregation process of relationship path similarity to generate a graphed vertical category knowledge base;
[0008] The government affairs data is learned using a pre-trained language model, and a vertical language model with vertical domain knowledge is obtained through masked language modeling and secondary language modeling; the parameters of the vertical language model are fine-tuned using the structured knowledge in the vertical knowledge base and the structured knowledge in the vertical knowledge base is integrated into the vertical language model, thereby obtaining a vertical government affairs large model that integrates the vertical knowledge base;
[0009] Obtain natural language government affairs queries input by users, understand and semantically parse the natural language government affairs queries through the vertical government affairs big model, and combine the graphed vertical knowledge base to perform knowledge reasoning and question answer generation to obtain natural language responses that meet user query needs; based on user feedback information during the question-and-answer dialogue process, interactively learn the vertical government affairs big model, and optimize the model parameters of the vertical government affairs big model by constructing a reward function for model optimization using a reinforcement learning algorithm.
[0010] In an optional embodiment,
[0011] Based on the ontology knowledge base, by constructing a graph data structure of multi-type nodes and multi-relationship edges, combining multiple relationship paths and the attention node aggregation process of relationship path similarity, the semantic associations between multi-type nodes are captured, and a graph-based vertical knowledge base is generated, including:
[0012] Define a set of node types of pre-constructed vertical heterogeneous graphs according to the concept hierarchy and instance types in the ontology knowledge base, use the attribute information of concept name, instance type, and text description in the ontology knowledge base as node attributes, define a set of edge types of pre-constructed vertical heterogeneous graphs according to the relationship types in the ontology knowledge base, extract the relationship strength in the ontology knowledge base as the edge attribute, and construct a vertical heterogeneous graph containing multi-type nodes and multi-relationship edges based on node types, node attributes, edge types, and edge attributes;
[0013] According to the node types and edge types in the vertical heterogeneous graph, a random walk algorithm is used to generate multiple relationship paths between nodes starting from each node according to the predefined meta-path pattern, and the node type, edge type, and aggregated features of node attributes and edge attributes on each relationship path are extracted to form a vector representation of the relationship path;
[0014] Using a sequence matching algorithm based on path structure and a semantic embedding model based on path semantics, the structural similarity and semantic similarity between different relationship paths are calculated, and a similarity matrix between the relationship paths is constructed based on the structural similarity and the semantic similarity;
[0015] Input the vector representation of each relationship path into the attention network, calculate the attention weight matrix between the relationship paths, perform weighted summation on the relationship paths based on the attention weight matrix to obtain the aggregate representation of the semantic association between nodes, and update the aggregate representation of the semantic association between nodes to the corresponding node attributes;
[0016] A semantic association graph is constructed according to the strength of semantic association between nodes, and the information of semantic association between multiple types of nodes is integrated to obtain a graph-based vertical knowledge base.
[0017] In an optional embodiment,
[0018] Using a sequence matching algorithm based on path structure and a semantic embedding model based on path semantics, calculating the structural similarity and semantic similarity between different relationship paths, and constructing a similarity matrix between relationship paths based on the structural similarity and the semantic similarity includes:
[0019] Each relationship path is represented as a path sequence of node type and edge type. The longest common subsequence length of two path sequences is calculated using the longest common subsequence algorithm. The path structure similarity based on the longest common subsequence is calculated based on the longest common subsequence length and the lengths of the two path sequences.
[0020] The edit distance algorithm is used to calculate the number of edit operations required to transform one path sequence into another path sequence, and the path structure similarity based on the edit distance is calculated according to the maximum value of the number of edit operations and the length of the two path sequences;
[0021] The nodes and edges in the relationship path are regarded as words, the semantic vectors of the nodes and edges are learned by maximizing the probability of the relationship path appearing, and the semantic similarity between the relationship paths is calculated using the vector distance metric;
[0022] According to the path structure similarity based on the longest common subsequence, the path structure similarity based on the edit distance and the semantic similarity, the comprehensive similarity is obtained by linear fusion through weighted average, and the similarity matrix between the relationship paths is constructed based on multiple comprehensive similarities.
[0023] In an optional embodiment,
[0024] The government affairs data is learned using a pre-trained language model, and a vertical language model with vertical domain knowledge is obtained through masked language modeling and secondary language modeling; the parameters of the vertical language model are fine-tuned using the structured knowledge in the vertical knowledge base and the structured knowledge in the vertical knowledge base is integrated into the vertical language model, and a vertical government affairs large model integrating the vertical knowledge base is obtained, including:
[0025] Masked language modeling is used to randomly mask some words in government data and train a pre-trained language model to predict the masked words;
[0026] Use secondary language modeling to train the pre-trained language model to predict the next word while predicting its context words, modeling the second-order relationship between words;
[0027] The pre-trained language model is trained through the entity-aware language modeling task to perform entity tagging and randomly mask the tagged entities; the pre-trained language model is trained through the relationship-aware language modeling task to construct a text sequence containing relationship triples and randomly mask the head entity or tail entity of the relationship triple;
[0028] The masked language modeling loss, secondary language modeling loss, entity-aware language modeling loss and relationship-aware language modeling loss are added according to the preset weights as the overall training goal of the vertical government affairs big model. Multiple losses are jointly optimized through iteration until the preset iteration conditions are met, thus completing the training of the vertical government affairs big model that integrates the vertical knowledge base.
[0029] In an optional embodiment,
[0030] The masked language modeling loss is described by the following formula:
[0031] ;
[0032] in, L MLM represents the masked language modeling loss, n Indicates the number of words, m i Indicates i mask indicator variables, x i Indicates i A hidden word, X Represents a text sequence in government data, p( x i / X ) Represents a text sequence in a given government data, predicting the i The probability of a masked word;
[0033] The secondary language modeling loss is shown in the following formula:
[0034] ;
[0035] in, L SLM represents the secondary language modeling loss, Indicates that before a given i words, predict the next word x i+1 The probability of Indicates that before a given i word and next word x i+1 , predict context words x j probability;
[0036] The entity-aware language modeling loss is given by the following formula:
[0037] ;
[0038] in, L ELM represents the entity-aware language modeling loss,e i Represents entity indicator variable;
[0039] The relation-aware language modeling loss is shown in the following formula:
[0040] ;
[0041] in, L RLM represents the relation-aware language modeling loss, T represents a set of relation triples, h , r , t They represent the head entity, the relationship between the head entity and the tail entity, and the tail entity respectively. represents the log probability of the head entity h given the relation r, the tail entity t and the separator SEP, It represents the log probability of the tail entity t given the head entity h, relation r and separator SEP.
[0042] In an optional embodiment,
[0043] Obtain natural language government affairs queries input by users, understand and semantically parse natural language government affairs queries through the vertical government affairs big model, and combine with the graphed vertical knowledge base to perform knowledge reasoning and question answer generation to obtain natural language responses that meet user query needs; interactively learn the vertical government affairs big model based on user feedback information during the question-and-answer dialogue process, construct a reward function for model optimization, estimate the gradient through the policy gradient algorithm, and use the stochastic gradient ascent algorithm to update the parameters of the vertical government affairs big model, including:
[0044] Obtain the natural language government affairs query input by the user, use the vertical government affairs big model to understand and semantically parse the natural language government affairs query, and obtain the query semantic vector corresponding to the natural language government affairs query through the self-attention mechanism and forward propagation process;
[0045] Match the query semantic vector with entities and relationships in the graphed vertical knowledge base, calculate the matching results between the query semantic vector and the entities and relationships in the graphed vertical knowledge base, and use multi-hop reasoning to determine the query subgraph corresponding to the query semantic vector in the graphed vertical knowledge base through a graph attention network;
[0046] Input the query subgraph into the Seq2Seq framework of the vertical government affairs big model, and generate a natural language response corresponding to the query subgraph through an encoder-decoder structure;
[0047] Collect user feedback information during the question-and-answer dialogue process, build a reward function for evaluating the quality of the answer, use the reward function to define the optimization objective, estimate the gradient through the policy gradient algorithm, and use the stochastic gradient ascent algorithm to update the parameters of the vertical government affairs model.
[0048] In an optional embodiment,
[0049] Construct a reward function for evaluating the quality of answers, use the reward function to define the model optimization goal, estimate the gradient through the policy gradient algorithm, and use the stochastic gradient ascent algorithm to update the parameters of the vertical government affairs model, including:
[0050] ;
[0051] ;
[0052] in, represents the gradient of the objective function J(θ) with respect to the agent’s policy parameter θ, represents the expected operator, which means that given the strategy π θ In the case of , we find the expected distribution of question-answer pairs (Q, A), π θ represents the policy function with parameter θ, which is used to indicate the Q Generate Answers A The probability distribution of Q Indicates the question raised by the user. A represents the answer generated by the agent, T represents the total number of rounds of question-answer interaction, Indicates that at time step t , given the problem Q t In the case of A t The log probability of θ The gradient of R t Represents the time step t The cumulative reward at the end of the conversation is used to evaluate the generated answer A t Quality, γ Represents the discount factor, with a value range of [0, 1], which is used to balance the weight of immediate rewards and future rewards. r(A t ) Indicates that at time step t , generate the answer A t Instant rewards received.
[0053] According to a second aspect of the embodiments of the present invention,
[0054] Provides a vertical government affairs large model service system based on interactive learning, including:
[0055] The first unit is used to obtain government data and user query log data in each vertical category, clean, annotate and structure the government data, build an ontology knowledge base in the vertical category, and based on the ontology knowledge base, capture the semantic associations between multi-type nodes and multi-relationship edges by combining multiple relationship paths and the attention node aggregation process of relationship path similarity to generate a graph-based vertical category knowledge base;
[0056] The second unit is used to learn the government data using a pre-trained language model, and obtain a vertical language model with vertical domain knowledge through masked language modeling and secondary language modeling; fine-tune the parameters of the vertical language model using the structured knowledge in the vertical knowledge base and integrate the structured knowledge in the vertical knowledge base into the vertical language model, so as to obtain a vertical government large model integrating the vertical knowledge base;
[0057] The third unit is used to obtain natural language government affairs queries input by users, understand and semantically parse natural language government affairs queries through the vertical government affairs big model, and combine the graphed vertical knowledge base to perform knowledge reasoning and question answer generation to obtain natural language responses that meet user query needs; based on user feedback information during the question-and-answer dialogue process, the vertical government affairs big model is interactively learned, and by constructing a reward function for model optimization, the model parameters of the vertical government affairs big model are optimized using the reinforcement learning algorithm.
[0058] According to a third aspect of the embodiments of the present invention,
[0059] An electronic device is provided, comprising:
[0060] processor;
[0061] a memory for storing processor-executable instructions;
[0062] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0063] According to a fourth aspect of the embodiments of the present invention,
[0064] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0065] By constructing a graphed vertical knowledge base, the present invention can effectively organize and represent structured knowledge in specific vertical fields, capture semantic associations between multi-type entity nodes, and provide rich background knowledge and reasoning capabilities for intelligent question and answer. Compared with traditional planar knowledge bases, graphed knowledge representation methods can better characterize the complex relationships between entities and improve the expressiveness and query efficiency of the knowledge base. By using a pre-trained language model to learn vertical domain corpora and integrating the structured knowledge of the vertical knowledge base, the present invention constructs a vertical government affairs model that has both domain knowledge and language comprehension capabilities. Compared with general language models, vertical government affairs models can better understand and generate domain-related natural languages, improving the accuracy and applicability of intelligent question and answer. At the same time, the integration of the knowledge base enhances the model's knowledge management and reasoning capabilities, enabling it to use structured knowledge to answer questions.
[0066] The present invention proposes an interactive learning framework, which continuously optimizes the vertical government affairs big model by utilizing user feedback information during the question-and-answer dialogue process. Compared with traditional static training methods, interactive learning can dynamically adapt to user needs, use user feedback to guide model learning, and improve the personalization and context relevance of the question-and-answer system. The construction of the reward function and the application of the reinforcement learning algorithm enable the model to learn the optimal question-and-answer strategy and generate more accurate, complete and satisfactory replies. The vertical government affairs big model service of the present invention can significantly improve the quality and efficiency of intelligent question-and-answer in the government affairs field. Through natural language interaction, users can easily obtain the required government affairs information and services, reducing the user's retrieval and waiting time. Accurate and personalized question-and-answer services can improve user satisfaction and trust in the government affairs system. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1 A flowchart of a vertical government affairs big model service method based on interactive learning according to an embodiment of the present invention;
[0068] Figure 2 This is a structural diagram of the vertical government affairs big model service system based on interactive learning in an embodiment of the present invention. DETAILED DESCRIPTION
[0069] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0070] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0071] Figure 1 Schematic diagram of the process of the vertical government affairs big model service method based on interactive learning in an embodiment of the present invention. Figure 1 As shown, the method includes:
[0072] S101. Obtain government data and user query log data in each vertical category, clean, annotate and structure the government data, build an ontology knowledge base in the vertical category, and based on the ontology knowledge base, capture the semantic associations between multiple types of nodes and multiple relationship edges by combining multiple relationship paths and the attention node aggregation process of relationship path similarity to generate a graphed vertical category knowledge base;
[0073] S102. Using the pre-trained language model to learn the government affairs data, through masked language modeling and secondary language modeling, a vertical language model with vertical domain knowledge is obtained; using the structured knowledge in the vertical knowledge base to fine-tune the parameters of the vertical language model and integrating the structured knowledge in the vertical knowledge base into the vertical language model, a vertical government affairs large model that integrates the vertical knowledge base is obtained;
[0074] S103. Obtain natural language government affairs queries input by users, understand and semantically parse the natural language government affairs queries through the vertical government affairs big model, and combine the graphed vertical knowledge base to perform knowledge reasoning and question answer generation to obtain natural language responses that meet user query needs; based on user feedback information during the question-and-answer dialogue process, interactively learn the vertical government affairs big model, and optimize the model parameters of the vertical government affairs big model by constructing a reward function for model optimization using a reinforcement learning algorithm.
[0075] By constructing a graphed vertical knowledge base, the present invention can effectively organize and represent structured knowledge in specific vertical fields, capture semantic associations between multi-type entity nodes, and provide rich background knowledge and reasoning capabilities for intelligent question and answer. Compared with traditional planar knowledge bases, graphed knowledge representation methods can better characterize the complex relationships between entities and improve the expressiveness and query efficiency of the knowledge base. By using a pre-trained language model to learn vertical domain corpora and integrating the structured knowledge of the vertical knowledge base, the present invention constructs a vertical government affairs model that has both domain knowledge and language comprehension capabilities. Compared with general language models, vertical government affairs models can better understand and generate domain-related natural languages, improving the accuracy and applicability of intelligent question and answer. At the same time, the integration of the knowledge base enhances the model's knowledge management and reasoning capabilities, enabling it to use structured knowledge to answer questions.
[0076] The present invention proposes an interactive learning framework, which continuously optimizes the vertical government affairs big model by utilizing user feedback information during the question-and-answer dialogue process. Compared with traditional static training methods, interactive learning can dynamically adapt to user needs, use user feedback to guide model learning, and improve the personalization and context relevance of the question-and-answer system. The construction of the reward function and the application of the reinforcement learning algorithm enable the model to learn the optimal question-and-answer strategy and generate more accurate, complete and satisfactory replies. The vertical government affairs big model service of the present invention can significantly improve the quality and efficiency of intelligent question-and-answer in the government affairs field. Through natural language interaction, users can easily obtain the required government affairs information and services, reducing the user's retrieval and waiting time. Accurate and personalized question-and-answer services can improve user satisfaction and trust in the government affairs system.
[0077] In an optional embodiment,
[0078] Based on the ontology knowledge base, by constructing a graph data structure of multi-type nodes and multi-relationship edges, combining multiple relationship paths and the attention node aggregation process of relationship path similarity, the semantic associations between multi-type nodes are captured, and a graph-based vertical knowledge base is generated, including:
[0079] Define a set of node types of pre-constructed vertical heterogeneous graphs according to the concept hierarchy and instance types in the ontology knowledge base, use the attribute information of concept name, instance type, and text description in the ontology knowledge base as node attributes, define a set of edge types of pre-constructed vertical heterogeneous graphs according to the relationship types in the ontology knowledge base, extract the relationship strength in the ontology knowledge base as the edge attribute, and construct a vertical heterogeneous graph containing multi-type nodes and multi-relationship edges based on node types, node attributes, edge types, and edge attributes;
[0080] According to the node types and edge types in the vertical heterogeneous graph, a random walk algorithm is used to generate multiple relationship paths between nodes starting from each node according to the predefined meta-path pattern, and the node type, edge type, and aggregated features of node attributes and edge attributes on each relationship path are extracted to form a vector representation of the relationship path;
[0081] Using a sequence matching algorithm based on path structure and a semantic embedding model based on path semantics, the structural similarity and semantic similarity between different relationship paths are calculated, and a similarity matrix between the relationship paths is constructed based on the structural similarity and the semantic similarity;
[0082] Input the vector representation of each relationship path into the attention network, calculate the attention weight matrix between the relationship paths, perform weighted summation on the relationship paths based on the attention weight matrix to obtain the aggregate representation of the semantic association between nodes, and update the aggregate representation of the semantic association between nodes to the corresponding node attributes;
[0083] A semantic association graph is constructed according to the strength of semantic association between nodes, and the information of semantic association between multiple types of nodes is integrated to obtain a graph-based vertical knowledge base.
[0084] Exemplarily, in order to capture the semantic associations between multi-type entities in a vertical knowledge base, it is first necessary to construct a vertical heterogeneous graph containing multi-type nodes and multi-relationship edges. Based on the concept hierarchy and instance type in the ontology knowledge base, a set of node types for the pre-built vertical heterogeneous graph is defined. Node types may include but are not limited to: concept nodes, instance nodes, attribute nodes, etc. Attribute information such as concept names, instance types, and text descriptions in the ontology knowledge base are used as node attributes to enrich the semantic representation of nodes.
[0085] According to the relationship types in the ontology knowledge base, define the edge type set of the pre-constructed vertical heterogeneous graph. Edge types may include but are not limited to: hierarchical relationship, synonymous relationship, attribute relationship, association relationship, etc. By analyzing the relationship strength in the ontology knowledge base, the relationship strength is extracted as the edge attribute to reflect the closeness of the association between different entity nodes.
[0086] Based on the defined node types, node attributes, edge types and edge attributes, a vertical heterogeneous graph containing multiple types of nodes and multiple relationship edges is constructed. The nodes and edges in the heterogeneous graph carry rich type and attribute information, providing a structured representation basis for subsequent semantic association capture.
[0087] In order to mine the semantic associations between nodes, it is necessary to generate multiple relationship paths between nodes. According to the node types and edge types in the vertical heterogeneous graph, the random walk algorithm is used to generate the relationship paths between nodes from each node according to the predefined meta-path pattern. The meta-path pattern defines the sequence of node types and edge types allowed to appear on the relationship path, reflecting the complex semantic association patterns between nodes of different types.
[0088] For each generated relationship path, the node type, edge type, and aggregated features of node attributes and edge attributes on the path are extracted to form a vector representation of the relationship path. Aggregated features can be calculated using methods such as average pooling, maximum pooling, and attention mechanism, taking into account the semantic information of different nodes and edges on the path. The vector representation of the relationship path facilitates the subsequent calculation of path similarity and node aggregation.
[0089] In order to measure the similarity between different relationship paths, a sequence matching algorithm based on path structure and a semantic embedding model based on path semantics are used to calculate the structural similarity and semantic similarity between relationship paths respectively.
[0090] The sequence matching algorithm based on path structure considers the sequence consistency of node types and edge types on the relationship path, and obtains the structural similarity between the paths by comparing the edit distance of the node type sequence and edge type sequence of the two paths. The higher the structural similarity, the closer the two paths are in structure.
[0091] The semantic embedding model based on path semantics maps the relationship path into a low-dimensional semantic space by learning the semantic representation of nodes and edges in the path. The semantic similarity between the paths is obtained by calculating the vector similarity of the relationship paths in the semantic space. The higher the semantic similarity, the closer the two paths are semantically.
[0092] Considering the structural similarity and semantic similarity, a similarity matrix between relationship paths is constructed. The similarity matrix reflects the degree of similarity between different relationship paths and provides a measure of path relevance for subsequent node aggregation.
[0093] In order to aggregate the information of semantic associations between nodes, the vector representation of each relationship path is input into the attention network, and the attention weight matrix between the relationship paths is calculated through the attention mechanism. The attention weight matrix reflects the importance of different relationship paths to node aggregation.
[0094] Based on the attention weight matrix, the relationship paths are weighted and summed to obtain the aggregate representation of the semantic association between nodes. The aggregate representation comprehensively considers the semantic information transmitted by different relationship paths and can more comprehensively characterize the semantic association between nodes. The aggregate representation of the semantic association between nodes is updated to the corresponding node attributes to enrich the semantic representation ability of the nodes.
[0095] The semantic association graph is constructed based on the strength of the semantic association between nodes. By setting the threshold of the semantic association strength, the strongly associated node pairs are screened out and the semantic association edges are generated. The edges in the semantic association graph reflect the semantic relevance between nodes, and the weight of the edge represents the strength of the semantic association.
[0096] Finally, the information of semantic associations between multiple types of nodes is integrated to obtain a graph-based vertical knowledge base. The graph-based knowledge base not only contains the concepts, instances, attributes, and relationships in the original ontology knowledge base, but also incorporates the semantic association information between nodes captured in the heterogeneous graph. The rich semantic associations provide more powerful support for the intelligent application of the knowledge base, such as knowledge reasoning, semantic search, and intelligent question and answer.
[0097] Through the above steps, a vertical heterogeneous graph containing multi-type nodes and multi-relationship edges was constructed based on the ontology knowledge base, the semantic associations between nodes were mined, and a graphed vertical knowledge base was generated. This method makes full use of the structured information in the ontology knowledge base, combined with the flexible representation capabilities of the heterogeneous graph, and effectively captures the semantic associations between multi-type nodes through random walks, path similarity calculations, and attention aggregation techniques, laying a solid foundation for the intelligent application of vertical knowledge bases.
[0098] In an optional embodiment,
[0099] Using a sequence matching algorithm based on path structure and a semantic embedding model based on path semantics, calculating the structural similarity and semantic similarity between different relationship paths, and constructing a similarity matrix between relationship paths based on the structural similarity and the semantic similarity includes:
[0100] Each relationship path is represented as a path sequence of node type and edge type. The longest common subsequence length of two path sequences is calculated using the longest common subsequence algorithm. The path structure similarity based on the longest common subsequence is calculated based on the longest common subsequence length and the lengths of the two path sequences.
[0101] The edit distance algorithm is used to calculate the number of edit operations required to transform one path sequence into another path sequence, and the path structure similarity based on the edit distance is calculated according to the maximum value of the number of edit operations and the length of the two path sequences;
[0102] The nodes and edges in the relationship path are regarded as words, the semantic vectors of the nodes and edges are learned by maximizing the probability of the relationship path appearing, and the semantic similarity between the relationship paths is calculated using the vector distance metric;
[0103] According to the path structure similarity based on the longest common subsequence, the path structure similarity based on the edit distance and the semantic similarity, the comprehensive similarity is obtained by linear fusion through weighted average, and the similarity matrix between the relationship paths is constructed based on multiple comprehensive similarities.
[0104] For example, in order to calculate the structural similarity between relationship paths, each relationship path is first represented as a path sequence of node types and edge types. The path sequence reflects the type order of nodes and edges on the relationship path and embodies the structural characteristics of the path.
[0105] The longest common subsequence algorithm is used to calculate the length of the longest common subsequence of two path sequences. The longest common subsequence refers to the longest subsequence that appears in the same order in the two sequences, but is not required to be continuous. The longer the length of the longest common subsequence, the higher the structural similarity of the two path sequences.
[0106] According to the length of the longest common subsequence and the length of the two path sequences, the path structure similarity based on the longest common subsequence is calculated. The calculation formula of structural similarity is: the length of the longest common subsequence divided by the arithmetic mean of the lengths of the two path sequences. The value range of this similarity is 0 to 1, and the closer to 1, the more similar the path structure is.
[0107] Another method to calculate the path structure similarity is to use the edit distance algorithm. The edit distance refers to the minimum number of edit operations required to transform one sequence into another sequence. Edit operations include insertion, deletion, and substitution. For the path sequences of two relationship paths, the edit distance algorithm is used to calculate the number of edit operations required to transform one path sequence into another path sequence. The fewer the number of edit operations, the more similar the two path sequences are in structure.
[0108] The path structure similarity based on edit distance is calculated based on the maximum number of edit operations and the length of the two path sequences. The formula for calculating the structural similarity is: 1 minus the number of edit operations divided by the maximum length of the two path sequences. The value range of this similarity is 0 to 1, and the closer it is to 1, the more similar the path structures are.
[0109] In order to calculate the semantic similarity between relation paths, the nodes and edges in the relation paths are regarded as words, and the semantic vectors of nodes and edges are learned by maximizing the probability of the relation paths appearing. The semantic vector captures the semantic information of nodes and edges by mapping them into a low-dimensional semantic space.
[0110] The idea of language model is adopted to optimize the semantic vectors of nodes and edges by maximizing the probability of relationship paths appearing. The goal is to make the semantic vectors of nodes and edges that appear together in the relationship paths closer in the semantic space.
[0111] After learning the semantic vectors of nodes and edges, the vector distance metric is used to calculate the semantic similarity between relationship paths. Commonly used vector distance metrics include cosine similarity and Euclidean distance. The higher the semantic similarity, the closer the two relationship paths are semantically.
[0112] In order to comprehensively consider the path structure similarity and semantic similarity, the path structure similarity based on the longest common subsequence, the path structure similarity based on the edit distance and the semantic similarity are linearly fused by weighted averaging to obtain the comprehensive similarity.
[0113] Set the weight parameters of structural similarity and semantic similarity to reflect their importance in the comprehensive similarity. The weight parameters can be obtained based on experience or through validation set tuning. The formula for calculating the comprehensive similarity is: the longest common subsequence similarity multiplied by weight 1, plus the edit distance similarity multiplied by weight 2, plus the semantic similarity multiplied by weight 3, and finally divided by the sum of the weights.
[0114] For all relationship paths between node pairs, calculate the comprehensive similarity between them and construct a similarity matrix between relationship paths. The similarity matrix is a symmetric matrix, and the elements on the main diagonal are 1, indicating that the similarity between each path and itself is 1. The larger the element value in the matrix, the more similar the corresponding two relationship paths are.
[0115] Through the above steps, the structural characteristics and semantic information of the relationship path are comprehensively considered, the similarity between different relationship paths is calculated, and the relationship path similarity matrix is constructed. The similarity matrix provides a measure of path relevance for subsequent node aggregation, so that when aggregating node semantic representation, the semantic information transmitted by different relationship paths can be more comprehensively considered, and the accuracy of capturing node semantic associations can be improved. At the same time, by introducing a semantic embedding model and using large-scale relationship path data to learn the semantic representation of nodes and edges, the dimension of path similarity calculation is expanded, and the representation ability of the similarity matrix is improved.
[0116] In an optional embodiment,
[0117] The government affairs data is learned using a pre-trained language model, and a vertical language model with vertical domain knowledge is obtained through masked language modeling and secondary language modeling; the parameters of the vertical language model are fine-tuned using the structured knowledge in the vertical knowledge base and the structured knowledge in the vertical knowledge base is integrated into the vertical language model, and a vertical government affairs large model integrating the vertical knowledge base is obtained, including:
[0118] Masked language modeling is used to randomly mask some words in government data and train a pre-trained language model to predict the masked words;
[0119] Use secondary language modeling to train the pre-trained language model to predict the next word while predicting its context words, modeling the second-order relationship between words;
[0120] The pre-trained language model is trained through the entity-aware language modeling task to perform entity tagging and randomly mask the tagged entities; the pre-trained language model is trained through the relationship-aware language modeling task to construct a text sequence containing relationship triples and randomly mask the head entity or tail entity of the relationship triple;
[0121] The masked language modeling loss, secondary language modeling loss, entity-aware language modeling loss and relationship-aware language modeling loss are added according to the preset weights as the overall training goal of the vertical government affairs big model. Multiple losses are jointly optimized through iteration until the preset iteration conditions are met, thus completing the training of the vertical government affairs big model that integrates the vertical knowledge base.
[0122] In an optional embodiment,
[0123] The masked language modeling loss is described by the following formula:
[0124] ;
[0125] in, L MLM represents the masked language modeling loss, n Indicates the number of words, m i Indicates i mask indicator variables, x i Indicates i A hidden word, X Represents a text sequence in government data, p( x i / X ) Represents a text sequence in a given government data, predicting the i The probability of a masked word;
[0126] The secondary language modeling loss is shown in the following formula:
[0127] ;
[0128] in, L SLM represents the secondary language modeling loss, Indicates that before a given i words, predict the next word x i+1 The probability of Indicates that before a given i word and next word x i+1, predict context words x j probability;
[0129] The entity-aware language modeling loss is given by the following formula:
[0130] ;
[0131] in, L ELM represents the entity-aware language modeling loss, e i Represents entity indicator variable;
[0132] The relation-aware language modeling loss is shown in the following formula:
[0133] ;
[0134] in, L RLM represents the relation-aware language modeling loss, T represents a set of relation triples, h , r , t They represent the head entity, the relationship between the head entity and the tail entity, and the tail entity respectively. represents the log probability of the head entity h given the relation r, the tail entity t and the separator SEP, It represents the log probability of the tail entity t given the head entity h, relation r and separator SEP.
[0135] For example, in order to train a vertical language model using unlabeled government data, the masked language modeling method is first used. Masked language modeling learns the contextual relationship and semantic information between words by randomly masking some words in the input text and training the language model to predict the masked words.
[0136] Specifically, for each input government text sequence, a certain proportion of words are randomly selected for masking. The masking operation includes replacing the selected words with special masking tags, keeping them unchanged, or replacing them with random words. The masked text sequence is used as the input of the language model, and the language model is required to predict the original words in the masked positions. By minimizing the cross-entropy loss of the predicted words, the language model learns to capture the contextual dependencies between words and model the language characteristics of government text.
[0137] In addition to masked language modeling, a secondary language modeling task is also used to enhance the language model's ability to model high-order relationships between words. Secondary language modeling requires the language model to predict the context words, that is, the words before and after the current word, while predicting the next word.
[0138] Specifically, for each input sequence of government text, the language model not only predicts the current word, but also predicts several words before and after the current word. By minimizing the joint prediction loss of the current word and its context words, the language model learns to capture the second-order relationship between words, that is, the dependency between a word and its context words. The secondary language modeling task forces the language model to model longer-range semantic dependencies and improve its ability to understand the semantics of government text.
[0139] In order to integrate the structured knowledge in the vertical knowledge base into the language model, the entity-aware language modeling task is introduced. Entity-aware language modeling aims to train the language model to identify and model entities in government texts.
[0140] Specifically, first, the government text is marked using named entity recognition tools or knowledge base matching methods to identify entity mentions in the text. Then, the marked entities are randomly masked, that is, the entities are replaced with special mask tags. The language model needs to predict the masked entities and learn the semantic relationship between the entity and its context by minimizing the cross entropy loss of entity prediction. The entity-aware language modeling task enables the language model to establish associations between vocabulary and entities and integrate entity information in the knowledge base.
[0141] In order to further introduce the relational knowledge in the knowledge base into the language model, a relation-aware language modeling task is adopted. Relation-aware language modeling aims to train the language model to model the relational triples contained in government texts.
[0142] Specifically, the relation triples in the vertical knowledge base are used to construct a text sequence containing the head entity, relation, and tail entity. For example, for the relation triple (entity A, relation R, entity B), the text sequence "entity A relation R entity B" is constructed. Then, the head entity or the tail entity in the relation triple is randomly masked, and the language model is required to predict the masked entity. By minimizing the cross entropy loss of relation triple prediction, the language model learns to capture the relational knowledge between entities and incorporates the structured relations in the knowledge base into the language representation.
[0143] In order to comprehensively utilize the advantages of multiple language modeling tasks, the masked language modeling loss, secondary language modeling loss, entity-aware language modeling loss and relationship-aware language modeling loss are added according to preset weights as the overall training target of the vertical government affairs model.
[0144] By iteratively optimizing multiple loss functions, the language model simultaneously learns the language features of government texts, high-order relations between words, entity knowledge, and relational knowledge. In each training step, a language modeling task is randomly selected, the corresponding loss function is calculated, and the model parameters are updated according to the gradient of the loss function. The alternating training of multiple language modeling tasks enables the language model to fully integrate domain knowledge and improve the ability to understand and generate government texts.
[0145] The training process continues until the preset iteration conditions are met, such as reaching the maximum number of iterations or the convergence of the loss function. By jointly optimizing multiple language modeling tasks, a large vertical government affairs model that integrates the vertical knowledge base is obtained. This model not only has powerful language understanding and generation capabilities, but also integrates domain knowledge and can better handle natural language tasks in the government affairs field.
[0146] Through the above steps, multiple language modeling tasks such as masked language modeling, secondary language modeling, entity-aware language modeling, and relationship-aware language modeling are used to train the pre-trained language model on large-scale government data to obtain a vertical language model with vertical domain knowledge. Then, the vertical language model is fine-tuned using the structured knowledge in the vertical knowledge base, and the entity and relationship knowledge in the knowledge base are integrated into the language model through entity-aware and relationship-aware language modeling tasks to obtain a large vertical government model that integrates the vertical knowledge base. This model improves the ability to understand and generate government text by jointly optimizing multiple language modeling tasks and comprehensively utilizing unlabeled government text data and structured domain knowledge, providing a strong language intelligence foundation for smart government applications.
[0147] In an optional embodiment,
[0148] Obtain natural language government affairs queries input by users, understand and semantically parse natural language government affairs queries through the vertical government affairs big model, and combine with the graphed vertical knowledge base to perform knowledge reasoning and question answer generation to obtain natural language responses that meet user query needs; interactively learn the vertical government affairs big model based on user feedback information during the question-and-answer dialogue process, construct a reward function for model optimization, estimate the gradient through the policy gradient algorithm, and use the stochastic gradient ascent algorithm to update the parameters of the vertical government affairs big model, including:
[0149] Obtain the natural language government affairs query input by the user, use the vertical government affairs big model to understand and semantically parse the natural language government affairs query, and obtain the query semantic vector corresponding to the natural language government affairs query through the self-attention mechanism and forward propagation process;
[0150] Match the query semantic vector with entities and relationships in the graphed vertical knowledge base, calculate the matching results between the query semantic vector and the entities and relationships in the graphed vertical knowledge base, and use multi-hop reasoning to determine the query subgraph corresponding to the query semantic vector in the graphed vertical knowledge base through a graph attention network;
[0151] Input the query subgraph into the Seq2Seq framework of the vertical government affairs big model, and generate a natural language response corresponding to the query subgraph through an encoder-decoder structure;
[0152] Collect user feedback information during the question-and-answer dialogue process, build a reward function for evaluating the quality of the answer, use the reward function to define the optimization objective, estimate the gradient through the policy gradient algorithm, and use the stochastic gradient ascent algorithm to update the parameters of the vertical government affairs model.
[0153] In an optional embodiment,
[0154] Construct a reward function for evaluating the quality of answers, use the reward function to define the model optimization goal, estimate the gradient through the policy gradient algorithm, and use the stochastic gradient ascent algorithm to update the parameters of the vertical government affairs model, including:
[0155] ;
[0156] ;
[0157] in, represents the gradient of the objective function J(θ) with respect to the agent’s policy parameter θ, represents the expected operator, which means that given the strategy π θ In the case of , we find the expected distribution of question-answer pairs (Q, A), π θ represents the policy function with parameter θ, which is used to indicate the Q Generate Answers A The probability distribution of Q Indicates the question raised by the user. A represents the answer generated by the agent, T represents the total number of rounds of question-answer interaction, Indicates that at time step t , given the problem Q t In the case of A t The log probability of θ The gradient of R t Represents the time step t The cumulative reward until the end of the conversation is used to evaluate the generated answer At Quality, γ Represents the discount factor, with a value range of [0, 1], which is used to balance the weight of immediate rewards and future rewards. r(A t ) Indicates that at time step t , generate the answer A t Instant rewards received.
[0158] For example, in order to realize intelligent government question answering, it is first necessary to understand and semantically parse the natural language government query input by the user. Using the trained vertical government model, through the self-attention mechanism and forward propagation process, the natural language query is mapped to the semantic space to obtain the semantic representation of the query.
[0159] Specifically, the natural language government affairs query input by the user is used as the input of the vertical government affairs big model, and the self-attention mechanism of the model is used to capture the dependencies and contextual information between the words in the query. The self-attention mechanism obtains the importance of the word in the query context by calculating the attention weight between each word and other words in the query. Then, through the forward propagation process, the word representation in the query is transformed through multiple layers of nonlinear transformation to obtain the semantic vector representation of the query. This semantic vector contains the semantic information of the query and the user's intention, providing a basis for subsequent knowledge reasoning and question answer generation.
[0160] In order to retrieve relevant knowledge from the graphed vertical knowledge base based on user queries, it is necessary to match the query semantic vector with the entities and relations in the knowledge base, and determine the knowledge subgraph related to the query through knowledge reasoning.
[0161] First, the semantic similarity between the query semantic vector and each entity and relationship in the graph-based vertical knowledge base is calculated. The similarity between the query semantic vector and the entity and relationship representation is measured by methods such as cosine similarity. Entity and relationship representations with higher similarity are more relevant to the query semantics.
[0162] Then, using multi-hop reasoning technology, search for knowledge subgraphs related to the query in the graphed vertical knowledge base. Starting from entities with high semantic similarity to the query, the graph attention network iteratively expands the entities and relationships connected to the current entity to generate knowledge subgraphs related to the query. The graph attention network determines the importance in the reasoning process by learning the attention weights between entities and relationships, and guides the generation direction of the subgraph. The multi-hop reasoning process continues until the generated subgraph reaches the preset number of hops or covers the key entities and relationships involved in the query.
[0163] Through the above steps, a knowledge subgraph related to the user's query semantics is obtained. This subgraph contains the core entities and relationships required to answer the query, providing knowledge support for subsequent question answer generation.
[0164] The Seq2Seq framework of the vertical government affairs big model is used to convert the query subgraph obtained by knowledge reasoning into question answers in natural language. The Seq2Seq framework consists of two parts: an encoder and a decoder, and realizes the conversion of knowledge to text through the encoding-decoding process.
[0165] First, the query subgraph is input into the encoder, which encodes the subgraph through a graph neural network and learns the representation of entities and relations in the subgraph. The graph neural network updates the representation of the entity by aggregating the neighbor information of the entity, capturing the structural and semantic information of the entity in the subgraph.
[0166] Then, the subgraph representation obtained by the encoder is used as the input of the decoder, and the decoder generates the answer words one by one through the attention mechanism and recurrent neural network. At each decoding step, the decoder calculates the attention weight based on the current generated words, subgraph representation and previously generated words to determine the key information in the subgraph. At the same time, the decoder uses recurrent neural network modeling to generate dependencies between words to ensure that the generated answers are smooth and coherent.
[0167] Through the collaborative work of encoder-decoder, the Seq2Seq framework converts the query subgraph into a natural language question answer. The generated answer contains the key information required for the query and is presented to the user in natural language to meet the user's query needs.
[0168] In order to further improve the question-answering performance of the vertical government affairs model, an interactive learning mechanism is introduced to optimize the model using user feedback information. During the question-answering dialogue, user feedback on the generated answers is collected, such as user satisfaction scores, follow-up questions, etc., to construct a reward function for evaluating the quality of the answer.
[0169] The reward function scores the generated answers based on user feedback. Answers with higher scores are of better quality and meet user needs; answers with lower scores are in need of improvement. The reward function is used to define the optimization goal, that is, to maximize the expected value of the reward function, to encourage the model to generate high-quality answers.
[0170] The policy gradient algorithm is used to estimate the gradient of the reward function with respect to the model parameters. The policy gradient algorithm uses Monte Carlo sampling to approximate the gradient of the reward function with respect to the model parameters and determine the direction of parameter update. Then, the stochastic gradient ascent algorithm is used to update the parameters of the vertical government affairs model based on the estimated gradient, and the model is optimized to generate higher quality answers.
[0171] The interactive learning process is ongoing, user feedback is continuously collected, reward functions are updated, and model parameters are optimized. Through multiple rounds of interaction and iterative optimization, the vertical government affairs model gradually adapts to user needs and preferences, generating more accurate, relevant, and satisfactory answers to questions.
[0172] Through the above steps, the intelligent government affairs question-and-answer function is realized using the vertical government affairs big model. After the user enters a natural language government affairs query, the model understands and semantically parses the query to obtain the query semantic vector. Then, through knowledge reasoning and subgraph generation, the knowledge related to the query is retrieved from the graphed vertical knowledge base. Using the Seq2Seq framework, the knowledge subgraph is converted into a natural language question answer to meet the user's query needs. At the same time, through interactive learning and model optimization, the question-and-answer performance of the vertical government affairs big model is continuously improved to provide users with more intelligent and efficient government services.
[0173] Figure 2 Schematic diagram of the structure of the vertical government affairs big model service system based on interactive learning in an embodiment of the present invention. Figure 2 As shown, the system comprises:
[0174] The first unit is used to obtain government data and user query log data in each vertical category, clean, annotate and structure the government data, build an ontology knowledge base in the vertical category, and based on the ontology knowledge base, capture the semantic associations between multi-type nodes and multi-relationship edges by combining multiple relationship paths and the attention node aggregation process of relationship path similarity to generate a graph-based vertical category knowledge base;
[0175] The second unit is used to learn the government data using a pre-trained language model, and obtain a vertical language model with vertical domain knowledge through masked language modeling and secondary language modeling; fine-tune the parameters of the vertical language model using the structured knowledge in the vertical knowledge base and integrate the structured knowledge in the vertical knowledge base into the vertical language model, so as to obtain a vertical government large model integrating the vertical knowledge base;
[0176] The third unit is used to obtain natural language government affairs queries input by users, understand and semantically parse natural language government affairs queries through the vertical government affairs big model, and combine the graphed vertical knowledge base to perform knowledge reasoning and question answer generation to obtain natural language responses that meet user query needs; based on user feedback information during the question-and-answer dialogue process, the vertical government affairs big model is interactively learned, and by constructing a reward function for model optimization, the model parameters of the vertical government affairs big model are optimized using the reinforcement learning algorithm.
[0177] According to a third aspect of the embodiments of the present invention,
[0178] An electronic device is provided, comprising:
[0179] processor;
[0180] a memory for storing processor-executable instructions;
[0181] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0182] A fourth aspect of the embodiments of the present invention is:
[0183] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0184] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.
[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A vertical government affairs big model service method based on interactive learning, characterized by: include: Obtain government data and user query log data for each vertical category, clean, annotate and structure the government data, build an ontology knowledge base for the vertical category, and based on the ontology knowledge base, capture the semantic associations between multiple types of nodes and multiple relationship edges by combining multiple relationship paths and the attention node aggregation process of relationship path similarity to generate a graphed vertical category knowledge base; The government affairs data is learned using a pre-trained language model, and a vertical language model with vertical domain knowledge is obtained through masked language modeling and secondary language modeling; the parameters of the vertical language model are fine-tuned using the structured knowledge in the vertical knowledge base and the structured knowledge in the vertical knowledge base is integrated into the vertical language model, thereby obtaining a vertical government affairs large model that integrates the vertical knowledge base; Obtain natural language government affairs queries input by users, understand and parse them semantically through the vertical government affairs big model, and perform knowledge reasoning and question answer generation in combination with the graphed vertical knowledge base to obtain natural language responses that meet the user's query needs; Based on user feedback during the question-and-answer dialogue, interactive learning is performed on the vertical government affairs big model. By constructing a reward function for model optimization, the model parameters of the vertical government affairs big model are optimized using a reinforcement learning algorithm. Based on the ontology knowledge base, by constructing a graph data structure of multi-type nodes and multi-relationship edges, combining multiple relationship paths and the attention node aggregation process of relationship path similarity, the semantic associations between multi-type nodes are captured, and a graph-based vertical knowledge base is generated, including: Define a set of node types of pre-constructed vertical heterogeneous graphs according to the concept hierarchy and instance types in the ontology knowledge base, use the attribute information of concept name, instance type, and text description in the ontology knowledge base as node attributes, define a set of edge types of pre-constructed vertical heterogeneous graphs according to the relationship types in the ontology knowledge base, extract the relationship strength in the ontology knowledge base as the edge attribute, and construct a vertical heterogeneous graph containing multi-type nodes and multi-relationship edges based on node types, node attributes, edge types, and edge attributes; According to the node types and edge types in the vertical heterogeneous graph, a random walk algorithm is used to generate multiple relationship paths between nodes starting from each node according to the predefined meta-path pattern, and the node type, edge type, and aggregated features of node attributes and edge attributes on each relationship path are extracted to form a vector representation of the relationship path; Using a sequence matching algorithm based on path structure and a semantic embedding model based on path semantics, the structural similarity and semantic similarity between different relationship paths are calculated, and a similarity matrix between the relationship paths is constructed based on the structural similarity and the semantic similarity; Input the vector representation of each relationship path into the attention network, calculate the attention weight matrix between the relationship paths, perform weighted summation on the relationship paths based on the attention weight matrix to obtain the aggregate representation of the semantic association between nodes, and update the aggregate representation of the semantic association between nodes to the corresponding node attributes; According to the strength of semantic associations between nodes, a semantic association graph is constructed, and information on semantic associations between multiple types of nodes is integrated to obtain a graph-based vertical knowledge base. The government affairs data is learned using a pre-trained language model, and a vertical language model with vertical domain knowledge is obtained through masked language modeling and secondary language modeling; the parameters of the vertical language model are fine-tuned using the structured knowledge in the vertical knowledge base and the structured knowledge in the vertical knowledge base is integrated into the vertical language model, and a vertical government affairs large model integrating the vertical knowledge base is obtained, including: Masked language modeling is used to randomly mask some words in government data and train a pre-trained language model to predict the masked words; Use secondary language modeling to train the pre-trained language model to predict the next word while predicting its context words, modeling the second-order relationship between words; The pre-trained language model is trained through the entity-aware language modeling task to perform entity tagging and randomly mask the tagged entities; the pre-trained language model is trained through the relationship-aware language modeling task to construct a text sequence containing relationship triples and randomly mask the head entity or tail entity of the relationship triple; The masked language modeling loss, secondary language modeling loss, entity-aware language modeling loss and relationship-aware language modeling loss are added according to the preset weights as the overall training goal of the vertical government affairs big model. Multiple losses are jointly optimized through iteration until the preset iteration conditions are met, thus completing the training of the vertical government affairs big model that integrates the vertical knowledge base.
2. The method according to claim 1, characterized in that Using a sequence matching algorithm based on path structure and a semantic embedding model based on path semantics, calculating the structural similarity and semantic similarity between different relationship paths, and constructing a similarity matrix between relationship paths based on the structural similarity and the semantic similarity includes: Each relationship path is represented as a path sequence of node type and edge type. The longest common subsequence length of two path sequences is calculated using the longest common subsequence algorithm. The path structure similarity based on the longest common subsequence is calculated based on the longest common subsequence length and the lengths of the two path sequences. The edit distance algorithm is used to calculate the number of edit operations required to transform one path sequence into another path sequence, and the path structure similarity based on the edit distance is calculated according to the maximum value of the number of edit operations and the length of the two path sequences; The nodes and edges in the relationship path are regarded as words, the semantic vectors of the nodes and edges are learned by maximizing the probability of the relationship path appearing, and the semantic similarity between the relationship paths is calculated using the vector distance metric; According to the path structure similarity based on the longest common subsequence, the path structure similarity based on the edit distance and the semantic similarity, the comprehensive similarity is obtained by linear fusion through weighted average, and the similarity matrix between the relationship paths is constructed based on multiple comprehensive similarities.
3. The method according to claim 1, characterized in that The masked language modeling loss is described by the following formula: ; Among them, L MLM represents the masked language modeling loss, n represents the number of words, and m i represents the i-th mask indicator variable, x i represents the i-th masked word, X represents the text sequence in the government data, p( x i / X ) represents the probability of predicting the i-th masked word given a text sequence in government data; The secondary language modeling loss is as follows: ; Among them, L SLM represents the secondary language modeling loss, It means given the previous i words, predict the next word x i+1 The probability of Indicates that given the previous i words and the next word x i+1 , predict the context word x j probability; The entity-aware language modeling loss is given by the following formula: ; Among them, L ELM represents the entity-aware language modeling loss, e i Represents entity indicator variable; The relation-aware language modeling loss is shown in the following formula: ; Among them, L RLM represents the relation-aware language modeling loss, T represents the set of relation triplets, h, r, and t represent the head entity, the relation between the head entity and the tail entity, and the tail entity, respectively. represents the log probability of the head entity h given the relation r, the tail entity t and the separator SEP, It represents the log probability of the tail entity t given the head entity h, relation r and separator SEP.
4. The method according to claim 1, characterized in that: Obtain natural language government affairs queries input by users, understand and parse them semantically through the vertical government affairs big model, and perform knowledge reasoning and question answer generation in combination with the graphed vertical knowledge base to obtain natural language responses that meet the user's query needs; Based on the user feedback information during the question-and-answer dialogue, the vertical government affairs big model is interactively learned. By constructing a reward function for model optimization, the gradient is estimated through the policy gradient algorithm and the parameters of the vertical government affairs big model are updated using the stochastic gradient ascent algorithm, including: Obtain the natural language government affairs query input by the user, use the vertical government affairs big model to understand and semantically parse the natural language government affairs query, and obtain the query semantic vector corresponding to the natural language government affairs query through the self-attention mechanism and forward propagation process; Match the query semantic vector with entities and relationships in the graphed vertical knowledge base, calculate the matching results between the query semantic vector and the entities and relationships in the graphed vertical knowledge base, and use multi-hop reasoning to determine the query subgraph corresponding to the query semantic vector in the graphed vertical knowledge base through a graph attention network; Input the query subgraph into the Seq2Seq framework of the vertical government affairs big model, and generate a natural language response corresponding to the query subgraph through an encoder-decoder structure; Collect user feedback information during the question-and-answer dialogue process, build a reward function for evaluating the quality of the answer, use the reward function to define the optimization objective, estimate the gradient through the policy gradient algorithm, and use the stochastic gradient ascent algorithm to update the parameters of the vertical government affairs model.
5. The method according to claim 4, characterized in that Construct a reward function for evaluating the quality of answers, use the reward function to define the model optimization goal, estimate the gradient through the policy gradient algorithm, and use the stochastic gradient ascent algorithm to update the parameters of the vertical government affairs model, including: ; ; in, represents the gradient of the objective function J(θ) with respect to the agent’s policy parameter θ, represents the expectation operator, which means that given the strategy π θ In the case of , we find the expected distribution of question-answer pairs (Q, A), π θ represents a policy function with parameter θ, which is used to indicate the probability distribution of generating an answer A based on a given question Q, where Q represents the question asked by the user, A represents the answer generated by the agent, and T represents the total number of rounds of question-answering interaction. Indicates that at time step t, given the problem Q t In the case of t The gradient of the log probability with respect to the parameter θ, R t represents the cumulative reward from time step t to the end of the dialogue, which is used to evaluate the generated answer A t The quality of, γ represents the discount factor, the value range is [0, 1], which is used to balance the weight of immediate rewards and future rewards, r(A t ) means that at time step t, the answer A is generated t Instant rewards received.
6. A vertical government affairs big model service system based on interactive learning, used to implement the method described in any one of claims 1 to 5, characterized in that: include: The first unit is used to obtain government data and user query log data in each vertical category, clean, annotate and structure the government data, build an ontology knowledge base in the vertical category, and based on the ontology knowledge base, capture the semantic associations between multi-type nodes and multi-relationship edges by combining multiple relationship paths and the attention node aggregation process of relationship path similarity to generate a graph-based vertical category knowledge base; The second unit is used to learn the government data using a pre-trained language model, and obtain a vertical language model with vertical domain knowledge through masked language modeling and secondary language modeling; fine-tune the parameters of the vertical language model using the structured knowledge in the vertical knowledge base and integrate the structured knowledge in the vertical knowledge base into the vertical language model, so as to obtain a vertical government large model integrating the vertical knowledge base; The third unit is used to obtain natural language government affairs queries input by users, understand and semantically analyze natural language government affairs queries through the vertical government affairs big model, and combine with the graphed vertical knowledge base to perform knowledge reasoning and generate question answers to obtain natural language responses that meet user query needs; Based on the user feedback information during the question-and-answer dialogue process, interactive learning is carried out on the vertical government affairs big model. By constructing a reward function for model optimization, the model parameters of the vertical government affairs big model are optimized using the reinforcement learning algorithm.
7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 5.
8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.
