A method, device and equipment for enhancing large model reasoning based on knowledge graph

Abstract

This specification provides a method, device and equipment for enhancing large model reasoning based on knowledge graph, by determining the matching nodes and target paths from a preset knowledge graph library according to the input entities in the input text of the input large model, and the predicted target relationship between the input entity and the answer to the question. Then, a prompt path is determined according to the matching nodes and the target path, so as to construct a subgraph according to the prompt path, determine the prompt text for describing the matching node, and input the large model. While supplementing the large model with effective knowledge, it avoids exceeding the window limit of the large model due to a large amount of input text, reduces the noise input to the large model, improves the accuracy of the answer inferred by the large model, and reduces the appearance of "hallucinations".

Description

Technical Field

[0001] This specification relates to the field of large models, and in particular to a method, device and equipment for enhancing large model reasoning based on knowledge graphs. Background Art

[0002] With the development of big model technology, big models have demonstrated their powerful functions in many fields, such as natural language processing, computer vision analysis, etc. However, since big models usually generate text based on statistical probability, when the big model generates output text based on the input text, if there is a lack of external knowledge sources, the big model cannot verify the authenticity of the generated content, resulting in the problem of "hallucination" in the big model, that is, when the big model determines the output text based on the input text, it may give wrong output text or give wrong explanations for the correct output text.

[0003] In the prior art, in order to reduce the probability of "hallucination" in a large model, it is usually chosen to provide the large model with more abundant knowledge to supplement the database of the large model, so as to help the large model improve the accuracy and reliability of the answer. For example, the large model is supplemented with more abundant text information, knowledge graph information, etc.

[0004] However, large models usually process texts, and knowledge graphs exist as structured data. It is difficult for large models to directly understand the knowledge in knowledge graphs, and the knowledge is relatively discrete. Directly inputting knowledge graphs into large models will cause the large models to accept too much useless knowledge, affecting the efficiency and quality of the reply texts generated by the large models. Therefore, how to avoid introducing too much noise while retrieving effective knowledge has become an urgent problem to be solved. Therefore, this specification provides a method, device, and equipment for enhancing large model reasoning based on knowledge graphs. Summary of the invention

[0005] This specification provides a method, device and equipment for enhanced large model reasoning based on knowledge graph to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] Obtaining input text of the input large model, and extracting input entities from the input text;

[0008] According to the input entity and the input text, predict the target relationship between the output text that the large model needs to output and the input entity;

[0009] Determine each node and the edge between each node in a preset knowledge graph library, match the input entity with each node to determine the matching node, and match the target relationship with each edge to determine the matching edge;

[0010] Determine a target path according to a path in the knowledge graph library that contains the matching edge and whose path length is less than a first preset length;

[0011] A path in the knowledge graph that contains the matching node and whose path length is less than the first preset length is taken as a candidate path, the candidate path is matched with the target path, at least one prompt path is determined, a subgraph is constructed according to the prompt path, a prompt text describing the matching node is determined according to the subgraph, and the prompt text and the input text are input into the large model.

[0012] Optionally, extracting input entities from the input text specifically includes:

[0013] Extracting multiple candidate entities from the input text;

[0014] A candidate entity is selected from the extracted candidate entities as an input entity.

[0015] Optionally, matching the input entity with each node to determine a matching node specifically includes:

[0016] Determine first similarities between the input entity and each of the nodes;

[0017] The nodes whose first similarity is greater than a first preset value are used as matching nodes.

[0018] Optionally, matching the target relationship with each edge to determine the matching edge specifically includes:

[0019] Determine the node relationship represented by the edge between the nodes in the knowledge graph library;

[0020] Determine respective second similarities between the target relationship and the node relationship represented by each edge;

[0021] The edge whose second similarity is greater than the second preset value is used as the matching edge.

[0022] Optionally, determining a target path according to a path in the knowledge graph library that contains the matching edge and whose path length is less than a first preset length specifically includes:

[0023] Determine, in the knowledge graph library, a path that contains the matching edge and has a path length less than a first preset length as a screening path;

[0024] From each of the screening paths, at least one target path is determined.

[0025] Optionally, at least one target path is determined from each screening path, specifically including:

[0026] Determine each candidate node pair according to each node in the knowledge graph library, and determine the optional path of each candidate node pair; and take the first and last nodes of each screening path as positive node pairs;

[0027] For each candidate node pair, determining the number of times each of the screening paths is included in the optional paths of the candidate node pair, and determining a feature vector of the candidate node pair according to the number of times;

[0028] The logistic regression model is trained by inputting the feature vector of each candidate node pair into the logistic regression model and based on the probability that the candidate node pair output by the logistic regression model belongs to a positive node pair;

[0029] According to the trained logistic regression model, the influence weights of each screening path on the probability are determined, and according to the size of the influence weights, the target path in each screening path is determined.

[0030] Optionally, determining each candidate node pair according to each node in the knowledge graph library specifically includes:

[0031] Replacing one node in each positive node pair with another node in the knowledge graph library to determine each negative node pair;

[0032] The negative node pairs and the positive node pairs are used as candidate node pairs.

[0033] Optionally, constructing a subgraph according to the prompt path specifically includes: determining the path length of the prompt path, and determining a second preset length according to the path length of the prompt path, wherein the second preset length is smaller than the path length of the prompt path; determining a path of the second preset length with the matching node as the starting point in the knowledge graph library as an extended path; and constructing a subgraph according to the prompt path and the extended path.

[0034] This specification provides a device for enhancing large model reasoning based on knowledge graph, including:

[0035] An acquisition module, used for acquiring input text of the input large model, and extracting input entities from the input text;

[0036] A prediction module, used for predicting the target relationship between the output text to be output by the large model and the input entity according to the input entity and the input text;

[0037] A determination module is used to determine each node and the edges between each node in a preset knowledge graph library, match the input entity with each node, determine the matching node, and match the target relationship with each edge, determine the matching edge;

[0038] A screening module, used to determine a target path according to a path in the knowledge graph library that contains the matching edge and whose path length is less than a first preset length;

[0039] A prompt module is used to take the path in the knowledge graph that contains the matching node and has a path length less than the first preset length as a candidate path, match the candidate path with the target path, determine at least one prompt path, build a subgraph according to the prompt path, determine a prompt text describing the matching node according to the subgraph, and input the prompt text and the input text into the large model.

[0040] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned enhanced large model reasoning method based on the knowledge graph.

[0041] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned enhanced large model reasoning method based on knowledge graph when executing the program.

[0042] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0043] In a method for enhancing large model reasoning based on a knowledge graph described in this specification, a matching node and a target path are determined from a preset knowledge graph library according to an input entity in an input text input to a large model and a predicted target relationship between the input entity and a question answer. Then, a prompt path is determined according to the matching node and the target path, so as to construct a subgraph according to the prompt path, determine a prompt text for describing the matching node, and input the large model.

[0044] It can be seen from the above method that by inputting the input text into the big model, the target path is screened in the preset knowledge graph library to construct a high-quality subgraph. While supplementing the big model with effective knowledge, it avoids exceeding the window limit of the big model due to the large amount of input text, reduces the noise input to the big model, improves the accuracy of the big model's reasoning answers, and reduces the occurrence of "hallucinations". BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0046] Figure 1A schematic diagram of a method for enhancing large model reasoning based on knowledge graph provided in this specification;

[0047] Figure 2 A schematic diagram of a process for determining a target path provided in this specification;

[0048] Figure 3 A schematic diagram of a process for determining a feature vector provided in this specification;

[0049] Figure 4 A schematic diagram of the same path provided for this specification;

[0050] Figure 5 A schematic diagram of a process for determining a prompt path provided in this specification;

[0051] Figure 6 A schematic diagram of an enhanced large model reasoning device based on a knowledge graph provided in this specification;

[0052] Figure 7 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0055] Figure 1 A schematic diagram of a method for enhancing large model reasoning based on knowledge graph provided in this specification, specifically:

[0056] S100: Obtain input text of the input large model, and extract input entities from the input text.

[0057] In one or more embodiments of the present specification, there is no restriction on which specific device executes the method of enhanced large model reasoning based on knowledge graph, for example, a mobile terminal and a server, etc. However, since subsequent steps involve operations such as entity extraction and similarity calculation, and these operations that require high computing resources and high permissions are generally executed by servers, this specification will also use the server executing the method of enhanced large model reasoning based on knowledge graph as an example for description, wherein the server can be a single device or composed of multiple devices, for example, a distributed server, and this specification does not impose any restrictions on this.

[0058] Since inputting the entire knowledge graph library into the large model as supplementary knowledge for the large model will introduce too much noise into the input of the large model, affecting the quality and efficiency of the large model in generating output text, the server can first filter the knowledge in the knowledge graph library to reduce the knowledge input into the large model. In order to determine the knowledge in the knowledge graph that is related to the input text of the large model, the server can first extract the entities in the input text so that the knowledge related to the input text can be extracted from the knowledge graph later.

[0059] Specifically, the server may first obtain the input text of the input large model, and then extract the entities of the input text as the input entities through a preset entity recognition algorithm. For example, if the input text is "What is the melting point of tungsten wire?", the input entities of the input text should be "tungsten wire" and "melting point".

[0060] Furthermore, in order to reduce the situation where more noise is introduced when extracting knowledge from the knowledge graph due to the large number of extracted input entities, the server can also reduce the number of extracted input entities when extracting entities, that is, first extract a number of candidate entities from the input text, and then select the core entity of the input text from each candidate entity as the extracted input entity. For example, the input text entered is "What is the melting point of tungsten wire?", and "tungsten wire" is the main object mentioned in the question. The core of the input text is to understand the specific properties related to "tungsten wire", that is, its melting point. Although "melting point" is also an important part of the question, it is actually an attribute of the inquiry, not the core entity in the question. Then the core entity of the input text should be "tungsten wire".

[0061] It should be noted that in one or more embodiments of the present specification, there is no limitation on the specific algorithm or tool used by the server to realize entity extraction and input entity extraction. The algorithm or tool may be LSTM-CRF, HanLP, etc., or other algorithms may be used, which can be set according to actual needs.

[0062] In one or more embodiments of this specification, there is no limitation on how the server specifically determines the core entity. It can randomly select one from the determined candidate entities as the input entity, or it can determine the weight of each candidate entity in the input text based on the semantics of the input text and the extracted candidate entities based on the trained large language model, such as ChatGPT, etc. It is also possible to determine the input entity from each candidate entity based on other trained models, such as using the quality of the output text of the large model for the input text as a reward, and completing the model through reinforcement learning training, etc. Since there are many methods that can be used, they are not listed here one by one, and can be set according to actual needs.

[0063] S102: According to the input entity and the input text, predict the target relationship between the output text that the large model needs to output and the input entity.

[0064] In order to reduce the knowledge extracted from the knowledge graph and prevent the large model from introducing too much noise when reasoning about the output text of the input text based on the knowledge extracted from the total knowledge graph, the server can also predict the target relationship between the answer of the input text and the input entity based on the extracted input entity, so that the knowledge related to the target relationship can be extracted from the knowledge graph based on the target relationship.

[0065] Specifically, the server may input the extracted input entity of the input text and the input text into a pre-trained relationship prediction model, and predict the target relationship between the output text of the input text and the input entity through the relationship prediction model. When the number of extracted input entities is multiple, the server may predict the relationship between the input entity and the output text for each extracted input entity. For example, the input text is "What is the melting point of tungsten wire?", and the extracted input entities are "tungsten wire" and "melting point". Then the target relationship can be predicted to be "attribute relationship" and "specific value".

[0066] It should be noted that in one or more embodiments of this specification, the server is not limited to which specific algorithm is used to predict the relationship between the answer text of the input text and the input entity, such as NLTK, StandardCoreNLP, etc., which can be set according to actual needs.

[0067] Furthermore, since, when predicting the target relationship, more than one target relationship may be determined for the same input entity, in order to reduce the situation where more noise is introduced when extracting knowledge from the knowledge graph due to the large number of extracted target relationships, the server can select the core relationship as the target relationship from each target relationship for the candidate relationship predicted based on each input entity, thereby reducing the determined target relationships, thereby reducing the noise of the input large model.

[0068] It should be noted that in one or more embodiments of this specification, the specific method used by the server to determine the core target relationship is not limited, and the core target relationship can be extracted through the trained large language model by setting prompt words. For example, input "Please extract the core target relationship of the input text from each target relationship.", each target relationship and the input text into the trained large language model. The server can also randomly extract a number of target relationships from the extracted target relationships as the core target relationships. Since there are many methods that can be used, this specification will not go into details here.

[0069] S104: Determine each node and the edges between each node in a preset knowledge graph library, match the input entity with each node to determine the matching node, and match the target relationship with each edge to determine the matching edge.

[0070] In order to filter out knowledge related to the input text from the knowledge graph, thereby providing prompt verification for the output text of the large model reasoning input text, the server can first match the determined input entities with the nodes in the knowledge graph library and the edges between the nodes, thereby filtering out some related entities and relationships from the nodes in the knowledge graph library and the edges between the nodes.

[0071] Specifically, the server first determines each node and each edge in the preset knowledge graph library, and then determines each first similarity between the input entity and each node, and uses the node whose first similarity is greater than a first preset value as a matching node. Then, it determines each second similarity between the target relationship and each edge, and uses the edge whose second similarity is greater than a second preset value as a matching edge.

[0072] It should be noted that in one or more embodiments of this specification, there is no restriction on the specific method used by the server to determine the first similarity between the input entity and each node, and the second similarity between the target relationship and each edge. The first similarity and the second similarity can be calculated by text similarity, or by converting the input entity, each node, the target relationship and each edge into a vector, and then calculating the similarity of the vector to determine the first similarity and the second similarity. Since there are many methods for calculating similarity, they are not listed here one by one, and can be set according to actual needs. Of course, when calculating the first similarity and the second similarity, the server can use the same calculation method or different calculation methods, and this specification does not limit this.

[0073] Furthermore, in one or more embodiments of the present specification, there is no limitation on the specific method used by the server to determine the first preset value and the second preset value. They can be set based on manual experience, or based on the idea of ​​reinforcement learning, with the quality of the output text output by the large model as a reward to adjust the first preset value and the second preset value. There are many methods that can be used, which will not be elaborated here and can be set according to actual needs.

[0074] In addition, the determined first preset value and the determined second preset value may be the same or different, and this specification does not limit this.

[0075] S106: Determine a target path according to a path in the knowledge graph library that contains the matching edge and whose path length is less than a first preset length.

[0076] Since the prompts determined only based on the nodes that have a direct relationship with the matching node cannot provide sufficient knowledge for the large model, the answer to the input text may also be related to the node that has an indirect relationship with the matching node. Therefore, in order to provide richer knowledge for the large model, the server can determine the relationship path with matching edges in the knowledge graph based on the determined matching edges. Then, in order to reduce the noise input to the large model, the relationship path with a path length less than the first preset length is determined from the filtered relationship paths, and the target path is selected from the relationship paths with a path length less than the first preset length.

[0077] Specifically, the server determines that there is a path in the knowledge graph library with matching edges and the path is less than the first preset length as a screening path, and then determines at least one target path from each screening path. It should be noted that when determining the screening path, the server can use a breadth-first search algorithm to determine the relationship path with a certain matching edge as the screening path. Then, when determining the target path from each screening path, the edges in each screening path can be scored, and then from each screening path, according to the size of the score, several screening paths with higher scores are selected from each screening path as the target path. It is also possible to screen each screening path based on reinforcement learning technology to determine the target path, and it can also be achieved through decision trees, random forests and other methods. This specification does not limit this and can be set according to actual needs. Of course, when screening the screening path by reinforcement learning, the server can use the quality of the output text generated by the large model as a reward.

[0078] like Figure 2 As shown, Figure 2 A schematic diagram of a process for determining a target path provided in this specification, wherein each circle represents a node in a knowledge graph library, a line segment represents an edge between two nodes in the knowledge graph library, and a thick line represents a matching edge in the knowledge graph library. Then, a path passing through the matching edge and having a path length less than a first preset length 3 is extracted as the target path.

[0079] S108: Take the path in the knowledge graph that contains the matching node and whose path length is less than the first preset length as a candidate path, match the candidate path with the target path, determine at least one prompt path, build a subgraph according to the prompt path, determine the prompt text describing the matching node according to the subgraph, and input the prompt text and the input text into the large model.

[0080] After determining the target path, the server can determine the hint path based on the determined target path and the matching nodes, and then build a subgraph based on the determined hint path to determine the hint text used to describe the matching nodes based on the subgraph, and then input the hint text and the input text into the big model to enrich the knowledge of the big model so that the big model can infer the answer to the input text based on the input hint text.

[0081] It should be noted that in one or more embodiments of the present specification, the server is not limited to a specific method for determining the prompt text used to describe the matching node based on the determined subgraph. It can be implemented through template filling method, rule-driven method, etc., or based on other methods, and can be set according to actual needs.

[0082] In addition, when determining the prompt text, the server can also determine a path of the second preset length starting from the matching node in the knowledge graph library as an extended path, and then construct a subgraph based on the prompt path and the extended path, thereby enriching the knowledge input into the large model.

[0083] Specifically, when determining the extension path, the server determines the path length of the extension path according to the determined path length of the prompt path, and then determines each extension path according to the path length of the extension path with the matching node as the initial point. Of course, when constructing a subgraph according to the prompt path and the extension path, the server should also determine each node on the prompt path and the extension path, and the constructed subgraph also includes the entities corresponding to the nodes in the knowledge graph.

[0084] It should be noted that, since the extended path is a path that provides additional knowledge for the large model in addition to the determined prompt path, when the path length of the extended path is greater than the path length of the prompt path, the determined extended path includes the prompt path determined based on the above method, and it is meaningless to determine the prompt path. Therefore, the path length of the extended path should be less than the path length of the prompt path.

[0085] When the path length of the prompt path is one hop, the server may no longer determine the extended path. Of course, the shortest path between a node that is one hop away from the matching node and the matching node may also be used as the extended path.

[0086] based on Figure 1 In a method for enhancing large model reasoning based on a knowledge graph, a matching node and a target path are determined from a preset knowledge graph library according to an input entity in an input text input to the large model and a predicted target relationship between the input entity and the answer to a question. Then, a prompt path is determined according to the matching node and the target path, a subgraph is constructed according to the prompt path, a prompt text for describing the matching node is determined, and the prompt text is input into the large model.

[0087] It can be seen from the above method that by inputting the input text into the big model, the target path is screened in the preset knowledge graph library to construct a high-quality subgraph. While supplementing the big model with effective knowledge, it avoids exceeding the window limit of the big model due to the large amount of input text, reduces the noise input to the big model, improves the accuracy of the big model's reasoning answers, and reduces the occurrence of "hallucinations".

[0088] In step S104, when the server calculates the first similarity or the second similarity by calculating vector similarity, the server may first determine the vector of each node and the vector of the edge between each node, and then calculate the second similarities between the vector of the input entity and the target relationship and the feature vector of each edge.

[0089] It should be noted that in one or more embodiments of the present specification, the server is not limited to the specific method used to convert the input entity and the target relationship into a vector, and the method used to determine the vector of each node and the vector of the preset relationship is the same. When determining the vector of each node and the vector of each edge, each node and each edge can be converted into a vector by word embedding, or by directly converting each node and each edge through a trained semantic model, or by converting each node and each edge into a vector through other algorithms or methods, which can be set according to actual needs.

[0090] The server can also pre-set a word vector library, which contains vectors corresponding to each word, letter, symbol, etc. When determining the vectors of the input entity, target relationship, each node, and each edge, the words, letters, symbols, etc. in the entity, edge, node, and target relationship are segmented, and the vectors corresponding to each word, letter, symbol, etc. in the word vector library are matched, and the matched vectors are combined as the feature vector corresponding to the entity or relationship. For example, when determining the feature vector of the input entity α, the name of the input entity α is divided into a word set α={ , ,……, }, then , ,……, Match with each vector in the word vector library to determine , ,……, , as the feature vector of the input entity α.

[0091] Of course, we can also average the word vectors and use the average vector as the feature vector of the input entity α, that is, , or it can be determined in other ways. This manual does not limit this and can be set according to actual needs.

[0092] If there is no preset entity with a first similarity greater than a first preset value and / or a preset relationship with a second similarity greater than a second preset value, it means that for the input text, the preset knowledge graph library cannot provide knowledge for the large model, and the server does not need to execute the steps after S106.

[0093] In step S106, the server can also filter each screening path through a logistic regression model, that is, the server determines each candidate node pair according to each node in the knowledge graph library, and determines the optional path between each candidate node pair. Of course, the length of the optional path is also less than the third preset length, and the third preset length can be the same as the first preset length, or slightly larger than the first preset length, and can be set according to actual needs. Then, for each candidate node pair, the feature vector of the candidate node pair is determined according to the number of times each screening path is included in the optional path of the candidate node pair. Then, the first and last nodes of each screening path are taken as positive node pairs, and the feature vectors of each candidate node pair are input into the logistic regression model. Based on the probability that the candidate node pair output by the logistic regression model belongs to the positive node pair, the logistic regression model is trained. Finally, the influence weight of each screening path on the probability is determined according to the trained logistic regression model, and the target path in each screening path is determined according to the size of the influence weight.

[0094] Among them, when constructing the feature vector, such as Figure 3 As shown, Figure 3 A schematic diagram of a process for determining a feature vector provided in this specification, wherein the determined screening path is ,in In the figure, the path "AB" is represented. represents the path "C", where In the path “CDE”, a candidate node pair The optional paths between include paths "AB", "ABCDE" and "ADE". Then the candidate node pair The eigenvector of is [2, 1, 1].

[0095] Furthermore, when determining the influence weight of each screening path on the probability according to the trained logistic regression model, the server may first determine the influence weight of each dimension of the input feature vector on the probability output by the model according to the trained logistic regression model. Since each dimension of the feature vector represents the corresponding screening path, and the number of times it is included in the optional path between the candidate node pair corresponding to the feature vector, the screening path corresponding to each influence weight may be determined according to the determined influence weight. Finally, according to the size of the influence weight, a preset number of screening paths with larger influence weights are selected as the target paths. It should be noted that when determining a preset number of target paths, the preset number can be set according to actual needs, and this manual does not impose any restrictions on this.

[0096] When determining candidate node pairs, in order to improve the training efficiency of the logistic regression model and thus improve the efficiency of generating reply texts of the large model, the server can determine nodes with higher correlation with matching edges from the nodes in the knowledge graph library as candidate node pairs, that is, the server replaces one node in the positive node pair with other nodes in the knowledge graph library to determine each negative node pair. Then the positive node pair and each negative node pair are used as candidate node pairs.

[0097] The following is an embodiment of determining the target path provided in this specification. Specifically, the server uses a breadth-first search algorithm to construct a positive node pair set S={ }, the matching edge can be used as a direct relationship between the two nodes in the node pair, or as an indirect relationship. Then determine the screening path set of the positive node pair set Then, for each node pair, one of the nodes is replaced with a random node in the knowledge graph to construct a negative node pair set. Each node pair in the positive node pair set and the negative node pair set is taken as a candidate node pair.

[0098] Then, the breadth-first search algorithm is used to determine the existence of For each candidate node pair, construct the corresponding feature vector , where the eigenvector Contains n dimensions, the i-th dimension represents The number of nodes in the set S and the set N that exist in the path between this pair of nodes.

[0099] It should be noted that in one or more embodiments of this specification, the path described refers to a path composed of edges, regardless of the nodes on the path, such as Figure 4 As shown, Figure 4 A schematic diagram of the same path provided for this specification, wherein a horizontal line represents the relationship between two nodes, and a circle represents a node. Although the entities represented by the nodes in these three paths are different, the relationships represented by the edges are the same and can be understood as the same path.

[0100] Next, when the candidate node belongs to the set S, the label of the candidate node is marked as 1, otherwise, it is marked as 0. The feature vector corresponding to each candidate node pair is Input the logistic regression model to be trained, so that the logistic regression model predicts the probability of the candidate node pair corresponding to the feature vector in the set S according to the input feature vector. Adjust the feature weight parameters of the logistic regression model, such as ,in is the feature weight vector to be optimized, is the total number of samples, is the label of the i-th sample, represents its eigenvector, is the probability that the label of the candidate node pair corresponding to the feature vector predicted by the model is 1. When the error between the output of the logistic regression model and the corresponding annotation meets the model accuracy requirement, or when the training of the logistic regression model reaches the preset rounds, or when the training of the logistic regression model reaches the preset duration, the training is stopped.

[0101] Finally, the feature weight parameters of the logistic regression model are determined, the dimensions corresponding to the features with the largest feature weight parameters are determined, and the corresponding relationship paths are determined according to the determined dimensions as the target path.

[0102] In step S108, when the server determines the hint path, the hint path and the target path refer to a path composed of edges, without considering the nodes on the path. Then, based on the relationship represented by the edges in the target path and the matching node, the paths of the matching node are searched to determine the hint path. Figure 5 As shown, Figure 5 This is a schematic diagram of a process for determining a prompt path provided in this specification. The black circle is a matching node, and the determined target path is "ABC". Then the determined prompt paths are 2, namely Figure 5 The prompt path 1 and the prompt path 2 are shown in .

[0103] Of course, when determining the prompt path, the prompt path can also be determined only with the matching node as the starting point. Figure 5 Taking the knowledge graph in as an example, the determined hint path should be hint path 2.

[0104] It should be noted that if the prompt path cannot be matched when determining the prompt path according to the target path, it can be determined that the knowledge graph library cannot provide knowledge for the large model. The server can mark the prompt text as empty.

[0105] The server may also first determine each path passing through the matching node, then calculate the similarity between each path and the target path, and use the path with a similarity greater than a threshold as a hint path. Further, the server may also use the matching node as the first node, determine each path with the matching node as the first node, then determine the similarity between each path with the matching node as the first node and the target path, and use the path with a similarity greater than a threshold as a hint path.

[0106] It should be noted that, in one or more embodiments of the present specification, if there is no path with a similarity greater than a threshold, the server may determine that the knowledge graph library cannot provide knowledge for the input text received by the large model.

[0107] The above is a method for enhancing large model reasoning based on knowledge graph provided by the embodiment of this specification. Based on the same idea, this specification also provides a corresponding device for enhancing large model reasoning based on knowledge graph, such as Figure 6 shown.

[0108] The acquisition module 400 is used to acquire the input text of the input large model and extract the input entity from the input text;

[0109] Prediction module 401, used for predicting the target relationship between the output text to be output by the large model and the input entity according to the input entity and the input text;

[0110] The determination module 402 is used to determine each node and the edges between each node in the preset knowledge graph library, match the input entity with each node to determine the matching node, and match the target relationship with each edge to determine the matching edge;

[0111] A screening module 403 determines a target path based on a path in the knowledge graph library that contains the matching edge and whose path length is less than a first preset length;

[0112] The prompt module 404 is used to take the path in the knowledge graph that contains the matching node and has a path length less than the first preset length as a candidate path, match the candidate path with the target path, determine at least one prompt path, build a subgraph according to the prompt path, determine the prompt text describing the matching node according to the subgraph, and input the prompt text and the input text into the large model.

[0113] Optionally, the acquisition module 400 is used to determine at least two candidate entities from the input text; and select one from the at least two candidate entities as the extracted input entity.

[0114] Optionally, the determination module 402 is used to determine first similarities between the input entity and each of the nodes respectively; and take nodes whose first similarities are greater than a first preset value as matching nodes.

[0115] Optionally, the determination module 402 is used to determine the node relationship represented by the edge between each node in the knowledge graph library; determine the second similarities between the target relationship and the node relationship represented by each edge; and use the edge whose second similarity is greater than a second preset value as a matching edge.

[0116] Optionally, the screening module 403 is used to determine a path in the knowledge graph library that contains the matching edge and has a path less than a first preset length as a screening path; and determine at least one target path from each screening path.

[0117] Optionally, the screening module 403 is used to determine each candidate node pair according to each node in the knowledge graph library, and determine the optional path of each candidate node pair; and, take the first and last nodes of each screening path as a positive node pair; for each candidate node pair, determine the number of times each screening path is included in the optional path of the candidate node pair, and determine the feature vector of the candidate node pair based on the number; input the feature vector of each candidate node pair into a logistic regression model, and train the logistic regression model based on the probability that the candidate node pair output by the logistic regression model belongs to a positive node pair; based on the trained logistic regression model, determine the influence weight of each screening path on the probability, and determine the target path in each screening path according to the size of the influence weight.

[0118] Optionally, the screening module 403 is used to replace one node in each positive node pair with other nodes in the knowledge graph library to determine each negative node pair; and use each negative node pair and each positive node pair as each candidate node pair.

[0119] Optionally, the prompt module 404 is used to determine the path length of the prompt path, determine a second preset length based on the path length of the prompt path, and the second preset length is smaller than the path length of the prompt path; determine a path of the second preset length with the matching node as the starting point in the knowledge graph library as an extended path; and construct a subgraph based on the prompt path and the extended path.

[0120] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0121] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0122] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provides data management methods.

[0123] This manual also provides Figure 7 As shown in FIG7 , at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The method of enhancing large model reasoning based on knowledge graph. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0124] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0125] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0126] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0127] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0128] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0132] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0133] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0134] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0135] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0136] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0138] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0139] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. A method for enhancing large model reasoning based on knowledge graph, characterized in that: include: Obtaining input text of the input large model, and extracting input entities from the input text; According to the input entity and the input text, predict the target relationship between the output text that the large model needs to output and the input entity; Determine each node and the edge between each node in a preset knowledge graph library, match the input entity with each node to determine the matching node, and match the target relationship with each edge to determine the matching edge; Determine a target path according to a path in the knowledge graph library that contains the matching edge and whose path length is less than a first preset length; A path in the knowledge graph that contains the matching node and whose path length is less than the first preset length is taken as a candidate path, the candidate path is matched with the target path, at least one prompt path is determined, a subgraph is constructed according to the prompt path, a prompt text describing the matching node is determined according to the subgraph, and the prompt text and the input text are input into the large model.

2. The method according to claim 1, characterized in that: Extracting input entities from the input text includes: Extracting multiple candidate entities from the input text; A candidate entity is selected from the extracted candidate entities as an input entity.

3. The method according to claim 1, characterized in that: Matching the input entity with each node to determine a matching node specifically includes: Determine first similarities between the input entity and each of the nodes; The nodes whose first similarity is greater than a first preset value are used as matching nodes.

4. The method according to claim 1, characterized in that: Matching the target relationship with each edge to determine the matching edge specifically includes: Determine the node relationship represented by the edge between the nodes in the knowledge graph library; Determine respective second similarities between the target relationship and the node relationship represented by each edge; The edge whose second similarity is greater than the second preset value is used as the matching edge.

5. The method according to claim 1, characterized in that Determining a target path according to a path in the knowledge graph library that contains the matching edge and whose path length is less than a first preset length specifically includes: Determine, in the knowledge graph library, a path that contains the matching edge and has a path length less than a first preset length as a screening path; From each of the screening paths, at least one target path is determined.

6. The method according to claim 5, characterized in that From each screening path, determine at least one target path, including: Determine each candidate node pair according to each node in the knowledge graph library, and determine the optional path of each candidate node pair; and take the first and last nodes of each screening path as positive node pairs; For each candidate node pair, determining the number of times each of the screening paths is included in the optional paths of the candidate node pair, and determining a feature vector of the candidate node pair according to the number of times; The logistic regression model is trained by inputting the feature vector of each candidate node pair into the logistic regression model and based on the probability that the candidate node pair output by the logistic regression model belongs to a positive node pair; According to the trained logistic regression model, the influence weights of each screening path on the probability are determined, and according to the size of the influence weights, the target path in each screening path is determined.

7. The method according to claim 6, characterized in that According to each node in the knowledge graph library, each candidate node pair is determined, specifically including: Replacing one node in each positive node pair with another node in the knowledge graph library to determine each negative node pair; The negative node pairs and the positive node pairs are used as candidate node pairs.

8. The method according to claim 1, characterized in that Constructing a subgraph according to the prompt path specifically includes: Determine a path length of the prompt path, and determine a second preset length according to the path length of the prompt path, wherein the second preset length is smaller than the path length of the prompt path; Determine, in the knowledge graph library, a path of a second preset length starting from the matching node as an extended path; A subgraph is constructed according to the hint path and the extended path.

9. A device for enhancing large model reasoning based on knowledge graph, characterized in that: include: An acquisition module, used for acquiring input text of the input large model, and extracting input entities from the input text; A prediction module, used for predicting the target relationship between the output text to be output by the large model and the input entity according to the input entity and the input text; A determination module is used to determine each node and the edges between each node in a preset knowledge graph library, match the input entity with each node, determine the matching node, and match the target relationship with each edge, determine the matching edge; A screening module, used to determine a target path according to a path in the knowledge graph library that contains the matching edge and whose path length is less than a first preset length; A prompt module is used to take the path in the knowledge graph that contains the matching node and has a path length less than the first preset length as a candidate path, match the candidate path with the target path, determine at least one prompt path, build a subgraph according to the prompt path, determine a prompt text describing the matching node according to the subgraph, and input the prompt text and the input text into the large model.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 8 is implemented.

11. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

Images