Knowledge query method, device and equipment based on large language model and storage medium

Abstract

The embodiment of the invention provides a knowledge query method and device based on a large language model, equipment and a storage medium. Comprising the steps of obtaining a to-be-queried statement associated with a target knowledge field and inputting the to-be-queried statement into a target large language model to obtain a knowledge query result; wherein the target large language model is obtained by performing difference minimization training on a pre-trained large language model based on a prediction sample query result and a real sample query result, and the prediction sample query result is generated by the pre-trained large language model based on a sample embedding vector and a sample indication statement; the sample embedding vector is obtained by aggregating the first embedding vector and the second embedding vector; the sample target path is determined according to the similarity between the sample to-be-queried statement and the multiple sample candidate paths; each sample candidate path is obtained by matching entities contained in the domain knowledge graph and association relationships among the entities according to a plurality of sample element paths. Therefore, the accuracy of the query result generated by the large language model can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a knowledge query method, apparatus, device and storage medium based on a large language model. Background Art

[0002] Large Language Model (LLM) is an artificial intelligence model built on deep learning technology, designed to understand and generate human language. These models are trained on massive text data to learn the statistical laws, structure and semantics of language, so that they can perform a variety of natural language processing tasks such as text classification, question answering, translation, dialogue, etc. However, although large language models perform well in conventional language, they still have the problem of "hallucination" in specific domain knowledge, resulting in inaccurate information generated by large language models.

[0003] In the training of large language models, in order to avoid the problem of "hallucination" in specific fields, knowledge in specific fields is usually introduced for training. However, during the training process, the model often generates answers through frequent queries. Such aimless queries not only consume a lot of computing power, but also introduce noise, which misleads the model to draw wrong conclusions, and then causes the trained model to have a lower accuracy when generating query results. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a knowledge query method, apparatus, device and storage medium based on a large language model, which can improve the accuracy of query results generated by the large language model.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a knowledge query method based on a large language model, the method comprising:

[0006] Obtain the query statements associated with the target knowledge domain;

[0007] Inputting the query sentence into the target large language model to obtain a knowledge query result;

[0008] The target large language model is obtained by performing difference minimization training based on a pre-trained large language model and predicted sample query results and real sample query results. The predicted sample query results are generated by the pre-trained large language model based on a sample embedding vector and a sample indication sentence. The sample embedding vector is obtained by aggregating a first embedding vector of a sample sentence to be queried and a second embedding vector of a description text corresponding to a sample target path. The sample indication sentence is obtained by concatenating the sample sentence to be queried with a query result of the sample sentence to be queried in a domain knowledge graph of the target knowledge domain.

[0009] Among them, the sample target path is determined according to the similarity between the sample query statement and multiple sample candidate paths; each sample candidate path is matched from the entities contained in the domain knowledge graph and the association relationships between the entities according to multiple sample meta-paths, and the multiple sample meta-paths are generated according to the sample association relationships between the sample entities in the sample query statement.

[0010] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a knowledge query device based on a large language model, the device comprising:

[0011] An acquisition module is used to acquire the query statements associated with the target knowledge domain;

[0012] An input module is used to input the query statement into a target large language model to obtain a knowledge query result; wherein the target large language model is obtained by performing difference minimization training based on a pre-trained large language model on the basis of predicted sample query results and real sample query results; the predicted sample query result is generated by the pre-trained large language model based on a sample embedding vector and a sample indication statement; the sample embedding vector is obtained by aggregating a first embedding vector of the sample query statement and a second embedding vector of a description text corresponding to a sample target path; the sample indication statement is obtained by splicing the sample query statement with a query result of the sample query statement in a domain knowledge graph of the target knowledge domain; wherein the sample target path is determined based on the similarity between the sample query statement and a plurality of sample candidate paths; each sample candidate path is obtained by matching the entities contained in the domain knowledge graph and the association relationships between the entities according to a plurality of sample meta-paths; the plurality of sample meta-paths are generated based on the sample association relationships between the sample entities in the sample query statement.

[0013] In some implementations, the large language model-based knowledge query apparatus further includes a training module for:

[0014] Acquire sample query statements associated with the target knowledge domain, and generate corresponding multiple sample meta-paths according to sample association relationships between sample entities in the query statements;

[0015] According to the multiple sample meta-paths, matching is performed from entities included in the domain knowledge graph and association relationships between the entities to obtain multiple sample candidate paths;

[0016] Determine a corresponding sample target path according to the similarity between the sample query statement and multiple sample candidate paths;

[0017] Aggregating the first embedding vector of the sample query sentence and the second embedding vector of the description text corresponding to the sample target path to obtain a sample embedding vector;

[0018] According to the sample query statement and the query result of the sample query statement in the domain knowledge graph of the target knowledge domain, a sample indication statement is obtained;

[0019] Generate a predicted sample query result based on the sample embedding vector and the sample indication sentence;

[0020] A real sample query result is obtained, and according to the predicted sample query result and the real sample query result, a difference minimization training is performed on the pre-trained large language model to obtain a target large language model.

[0021] In some embodiments, the training module is further used to:

[0022] According to entity types in the multiple sample meta-paths, determine multiple sample first entities of the multiple sample meta-paths from multiple entities included in the domain knowledge graph;

[0023] For each sample meta-path, repeatedly matching the next sample first entity according to the association relationship between the sample first entity and other entities to obtain an updated sample first entity, until the first entity type of the updated sample first entity is the last entity type in the corresponding each sample meta-path;

[0024] According to the plurality of sample first entities and the plurality of updated sample first entities corresponding to the plurality of sample meta-paths, a plurality of corresponding sample candidate paths are sequentially constructed.

[0025] In some embodiments, the training module is further used to:

[0026] The control object agent obtains a plurality of candidate sample entities according to the association relationship between the sample first entity and other entities;

[0027] Outputting the predicted probability of each candidate sample entity being the first entity of the next sample through the policy network of the object agent intelligent body;

[0028] Based on the multiple predicted probabilities of the multiple candidate sample entities, the object proxy agent is controlled to maximize the path matching reward, and an updated sample first entity is determined from the multiple candidate predicted probabilities.

[0029] In some embodiments, the training module is further used to:

[0030] Obtaining, through the policy network of the object proxy agent, a third embedding vector of the current sample first entity and a fourth embedding vector of the sample first entity at a previous time step of the current sample first entity;

[0031] The hidden state of the policy network is updated according to the third embedding vector and the fourth embedding vector, and the predicted probability of each candidate sample entity being the first entity of the next sample is output according to the updated hidden state.

[0032] In some embodiments, the training module is further used to:

[0033] After the object proxy agent reaches the updated sample first entity, the first entity type of the updated sample first entity is compared with the indicated entity type in the corresponding sample meta-path, and a first reward index for the object proxy agent is determined according to the comparison result;

[0034] Obtaining a cumulative path confidence score from a first sample first entity generated according to the sample meta-path to the updated sample first entity, and a path matching degree between a current path constructed from the first sample first entity to the updated sample first entity and a corresponding indicated entity type of the sample meta-path;

[0035] Obtaining a first reward amplitude parameter and a second reward amplitude parameter, and obtaining a second reward index according to the product of the first reward amplitude parameter, the second reward amplitude parameter, the path confidence score, and the path matching degree;

[0036] According to the sum of the first reward index and the second reward index, a path matching reward for the object agent to reach the updated sample first entity is determined.

[0037] In some embodiments, the training module is further used to:

[0038] Obtaining a path matching reward obtained after determining the first entity of each sample when the object proxy agent generates each sample candidate path, and a gradient contribution to the policy parameters of the policy network;

[0039] Obtaining a first gradient parameter by multiplying each of the path matching rewards by the corresponding gradient contribution;

[0040] Adding a plurality of first gradient parameters corresponding to the plurality of sample first entities to obtain a comprehensive gradient parameter;

[0041] The expected value of the comprehensive gradient parameter is obtained, and the parameters of the policy network are updated according to the expected value to obtain an updated policy network.

[0042] To achieve the above-mentioned purpose, the third aspect of the embodiments of the present application proposes a computer device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the knowledge query method based on a large language model as described in any one of the embodiments of the first aspect of the present application.

[0043] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the knowledge query method based on a large language model as described in any one of the embodiments of the first aspect of the present application.

[0044] The embodiment of the present application obtains the query sentence associated with the target knowledge domain; inputs the query sentence into the target large language model to obtain the knowledge query result; wherein, the target large language model is obtained by minimizing the difference training based on the predicted sample query result and the real sample query result by the pre-trained large language model, and the predicted sample query result is generated by the pre-trained large language model based on the sample embedding vector and the sample indication sentence, the sample embedding vector is obtained by aggregating the first embedding vector of the sample query sentence and the second embedding vector of the description text corresponding to the sample target path, and the sample indication sentence is obtained by splicing the query result of the sample query sentence and the sample query sentence in the domain knowledge graph of the target knowledge domain; wherein, the sample target path is determined according to the similarity between the sample query sentence and multiple sample candidate paths; each sample candidate path is matched from the entities contained in the domain knowledge graph and the association relationship between the entities according to multiple sample meta-paths, and multiple sample meta-paths are generated according to the sample association relationship between each sample entity in the sample query sentence. In this way, by predefining multiple sample meta-paths, the real sample candidate path can be retrieved in the domain knowledge graph, invalid search paths can be quickly excluded, computing power can be saved, and the influence of noise can be effectively avoided. At the same time, by calculating the similarity, the sample target path with the highest similarity to the sample query statement is determined, the question and the answer can be aligned, and a path search result that highly matches the user's intention can be generated. In addition, by inputting the sample embedding vector and the sample indication statement into the pre-trained large language model for fine-tuning, additional clues can be provided to the pre-trained large language model to help it better locate the core of the problem, while significantly improving the learning effect of the large language model in the target knowledge field and the accuracy of knowledge matching, thereby preventing the trained target large language model from answering query statements in the target knowledge field. Hallucination problems. In summary, the present application can improve the accuracy of query results generated by a trained target large language model by performing efficient and accurate knowledge enhancement training on the pre-trained large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the architecture of a knowledge query system based on a large language model provided in an embodiment of the present application;

[0046] Figure 2 is a flow chart of a knowledge query method based on a large language model provided in an embodiment of the present application;

[0047] Figure 3 It is a flowchart of training a pre-trained large language model provided in an embodiment of the present application;

[0048] Figure 4 It is an example diagram of the summary knowledge graph provided in the embodiment of the present application;

[0049] Figure 5 It is an overall flow chart of the knowledge query method based on a large language model provided in an embodiment of the present application;

[0050] Figure 6 It is a functional module diagram of a knowledge query device based on a large language model provided in an embodiment of the present application;

[0051] Figure 7 It is a schematic diagram of the hardware structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0055] Large Language Model (LLM) is an artificial intelligence model built on deep learning technology, designed to understand and generate human language. These models are trained on massive text data to learn the statistical laws, structure and semantics of language, so that they can perform a variety of natural language processing tasks such as text classification, question answering, translation, dialogue, etc. However, although large language models perform well in conventional language, they still have the problem of "hallucination" in specific domain knowledge, resulting in inaccurate information generated by large language models.

[0056] In the training of large language models, in order to avoid the problem of "hallucination" in specific fields, knowledge in specific fields is usually introduced for training. However, during the training process, the model often generates answers through frequent queries. Such aimless queries not only consume a lot of computing power, but also introduce noise, which misleads the model to draw wrong conclusions, and then causes the trained model to have a lower accuracy when generating query results.

[0057] Based on this, the embodiments of the present application provide a knowledge query method, apparatus, device and storage medium based on a large language model, which can improve the accuracy of query results generated by a trained target large language model.

[0058] The knowledge query method, apparatus, device and storage medium based on a large language model provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the knowledge query system based on a large language model in the embodiments of the present application is described.

[0059] Please refer to Figure 1 In some implementations, an embodiment of the present application provides a knowledge query system based on a large language model, and the knowledge query system includes a terminal 11 and a server 12.

[0060] Exemplarily, the terminal 11 may be a device directly operated by a user, such as a smart phone, a tablet computer, or a personal computer. Specifically, the terminal 11 may provide a user-friendly interface for the user to input a sentence to be queried, and perform preliminary processing on the original text input by the user, such as word segmentation, grammar checking, etc., package the processed sentence to be queried and send it to the server 12 via the network, and receive the knowledge query result returned by the server 12. Furthermore, the terminal may also parse the result returned by the server 12 and present it on the user interface in an easy-to-understand manner.

[0061] Exemplarily, the server end 12 may be a service system running in a remote data center or a cloud environment, etc. Specifically, the server end 12 may receive a query statement from the terminal 11, analyze and understand the query statement, determine the target knowledge domain corresponding to the query statement, and use a large language model to deeply understand and reason the query, generate a corresponding knowledge query result, and return the knowledge query result to the terminal 11.

[0062] The knowledge query method based on a large language model in the embodiment of the present application can be illustrated by the following embodiment.

[0063] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first. Moreover, the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or separate consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or separate consent, the necessary user-related data for enabling the normal operation of the embodiment of the present application will be obtained.

[0064] In the embodiment of the present application, the description will be made from the perspective of a knowledge query device based on a large language model, which can be integrated into a computer device. Figure 2 , Figure 2 The flowchart of the steps of the knowledge query method based on a large language model provided in the embodiment of the present application is as follows: The embodiment of the present application takes the knowledge query device based on a large language model as an example, which is specifically integrated on a terminal or a server. When the processor on the terminal or the server executes the program instructions corresponding to the knowledge query method based on a large language model, the specific process is as follows:

[0065] Step 101: Obtain the query statement associated with the target knowledge domain.

[0066] In some implementations, in order to help the system clarify the user's query intention, the query statements associated with the target knowledge field may be obtained to generate high-quality knowledge query results.

[0067] The target knowledge domain may be a specific domain or professional category involved in the query statement.

[0068] The query statement may be a specific question or query content input by the user, which is used to express the information or answer that the user expects to obtain. The query statement may be a natural language text, which may be a phrase, sentence or paragraph.

[0069] For example, the target knowledge domain may be the medical field, the financial field, the legal field, the scientific field, etc. For example, in the medical field, the query statement may be "Which genes are related to gemcitabine resistance in lung cancer treatment?"; in the scientific field, the query statement may be "What breakthroughs have been made in the latest quantum computing research?", etc.

[0070] In some embodiments, the corresponding target knowledge domain can be inputted while inputting the query statement. The query statement can also be analyzed to quickly locate the corresponding target knowledge domain, so that the system can better understand the user's query intention. For example, if the query statement is "Which genes are related to gemcitabine resistance in lung cancer treatment?", through analysis, it can be determined that the keywords corresponding to the query statement belong to the medical field.

[0071] By clarifying the target knowledge domain and query statements, the target large language model can more accurately parse the user's query intention, quickly utilize the domain knowledge graph and pre-trained parameters of a specific field, and generate higher quality knowledge query results.

[0072] Step 102, inputting the query sentence into the target large language model to obtain the knowledge query result;

[0073] The target large language model is obtained by performing difference minimization training based on the predicted sample query results and the real sample query results by the pre-trained large language model, the predicted sample query results are generated by the pre-trained large language model based on the sample embedding vector and the sample indication sentence, the sample embedding vector is obtained by aggregating the first embedding vector of the sample sentence to be queried and the second embedding vector of the description text corresponding to the sample target path, and the sample indication sentence is obtained by splicing the query results of the sample sentence to be queried and the sample sentence to be queried in the domain knowledge graph of the target knowledge domain;

[0074] Among them, the sample target path is determined according to the similarity between the sample query statement and multiple sample candidate paths; each sample candidate path is matched from the entities contained in the domain knowledge graph and the association relationships between entities according to multiple sample meta-paths, and multiple sample meta-paths are generated according to the sample association relationships between the sample entities in the sample query statement.

[0075] In some implementations, in order to fully utilize the finely tuned target large language model, the query statement may be input into the target large language model to obtain a knowledge query result, so as to generate high-quality query results.

[0076] The target large language model may be a large language model that has been trained and optimized with knowledge in a specific domain and can directly generate high-quality query results.

[0077] The knowledge query result can be the final answer generated by the target large language model according to the query sentence. The accuracy of the knowledge query result can be used to reflect the generation ability of the target large language model and the correctness of the information learned by the target large language model in the domain knowledge graph.

[0078] Among them, the pre-trained large language model can be a basic model, which is used to perform specific training on different domain knowledge graphs to obtain a target large language model.

[0079] The predicted sample query result may be a preliminary answer generated by the pre-trained large language model for the sample query sentence. The predicted sample query result is generated based on the current parameters and learned knowledge of the pre-trained large language model.

[0080] Among them, the real sample query results can be the correct answers obtained from actual data. The real sample query results are verified and can be used as a benchmark to evaluate and optimize the generation results of the pre-trained large language model.

[0081] Among them, the sample embedding vector can be a description text of the sample query sentence and the sample target path converted into an embedding vector representation of a fixed length, which is used to capture the semantic features of the sample query sentence and the sample target path, so that the pre-trained large language model can better understand and learn.

[0082] The sample indication sentence can be a sentence formed by splicing the sample query sentence and the query result obtained from the domain knowledge graph. The sample indication sentence provides more contextual information, which can help the pre-trained large language model better understand the background and intent of the query.

[0083] The sample query sentence may be a specific question or query content used to train a pre-trained large language model.

[0084] The first embedding vector may be a vector representation of a fixed length converted from the sample sentence to be queried, so as to capture the semantic features of the sample sentence to be queried.

[0085] The sample target path may be a path selected from a plurality of sample candidate paths that best matches the sample query statement, that is, a path with the greatest similarity to the sample query statement.

[0086] The second embedding vector may be a vector representation obtained by converting the description text of the sample target path into a fixed-length vector, and the second embedding vector is used to capture the semantic features of the sample target path.

[0087] Among them, the sample candidate paths can be multiple possible paths generated from the knowledge graph according to the sample entities in the sample query sentence, which are used to represent the various answers that may be generated according to the sample query sentence.

[0088] Among them, the sample meta-path can be a relationship path template between entities extracted from the sample query statement, which is used to guide the generation of sample candidate paths from the domain knowledge graph.

[0089] The sample association relationship may be the relationship between entities in the sample query statement.

[0090] For example, since the target large language model has learned a large amount of domain knowledge through a specific training process and has the ability to generate high-quality answers, the query statement can be input into the target large language model, and the target large language model can directly output the corresponding knowledge query result.

[0091] The embodiment of the present application obtains the query sentence associated with the target knowledge domain; inputs the query sentence into the target large language model to obtain the knowledge query result; wherein, the target large language model is obtained by minimizing the difference training based on the predicted sample query result and the real sample query result by the pre-trained large language model, and the predicted sample query result is generated by the pre-trained large language model based on the sample embedding vector and the sample indication sentence, the sample embedding vector is obtained by aggregating the first embedding vector of the sample query sentence and the second embedding vector of the description text corresponding to the sample target path, and the sample indication sentence is obtained by splicing the query result of the sample query sentence and the sample query sentence in the domain knowledge graph of the target knowledge domain; wherein, the sample target path is determined according to the similarity between the sample query sentence and multiple sample candidate paths; each sample candidate path is matched from the entities contained in the domain knowledge graph and the association relationship between the entities according to multiple sample meta-paths, and multiple sample meta-paths are generated according to the sample association relationship between each sample entity in the sample query sentence. In this way, by predefining multiple sample meta-paths, the real sample candidate path can be retrieved in the domain knowledge graph, invalid search paths can be quickly excluded, computing power can be saved, and the influence of noise can be effectively avoided. At the same time, by calculating the similarity, the sample target path with the highest similarity to the sample query statement is determined, the question and the answer can be aligned, and a path search result that highly matches the user's intention can be generated. In addition, by inputting the sample embedding vector and the sample indication statement into the pre-trained large language model for fine-tuning, additional clues can be provided to the pre-trained large language model to help it better locate the core of the problem, while significantly improving the learning effect of the large language model in the target knowledge field and the accuracy of knowledge matching, thereby preventing the trained target large language model from answering query statements in the target knowledge field. Hallucination problems. In summary, the present application can improve the accuracy of query results generated by a trained target large language model by performing efficient and accurate knowledge enhancement training on the pre-trained large language model.

[0092] Please refer to Figure 3 In some implementations, in order to enable the target large language model to generate accurate responses in a specific domain, the pre-trained large language model can be specifically trained through the corresponding domain knowledge graph, so that the pre-trained large language model learns the knowledge of the domain knowledge graph and improves its adaptability and accuracy in the domain. For example, the target large language model is trained in the following way:

[0093] Step 201 : obtaining sample query statements associated with the target knowledge domain, and generating corresponding multiple sample meta-paths according to sample association relationships between sample entities in the query statements.

[0094] Please refer to Figure 4For example, based on the experience and knowledge of domain experts, a summary knowledge graph for the corresponding domain knowledge graph can be defined, and the summary knowledge graph shows the relationship between related entities in the corresponding domain. For example, in one of the summary knowledge graphs in the medical field, sample entity types can be used as nodes, and sample associations between sample entity types can be used as edges, thereby showing the direction of information flow and reflecting the causal or influence relationship between entities. In this way, by mapping the sample entity types of the sample query statement to the summary knowledge graph, multiple sample meta-paths can be generated.

[0095] Exemplarily, natural language processing technology can be used to extract sample entities from sample query statements and determine which entity type these sample entities belong to. For example, "lung cancer" can belong to "disease" and "gemcitabine" can be "drug". After that, search for nodes that match the entity type in the summary knowledge graph, and start exploring along the relationship chain in the summary knowledge graph to generate a sample meta-path that conforms to the relationship chain. The sample meta-path includes a sample head entity and a sample tail entity and multiple relationship sets between the sample head entity and the sample tail entity. For example, a sample meta-path can be generated from lung cancer to therapeutic drugs, to gemcitabine, to influencing genes, to genes, and so on.

[0096] Furthermore, multiple sample meta-paths can also be generated by inputting multiple sample entities corresponding to the sample query statements into a pre-trained model, or can be set by technicians according to the query statements. The embodiments of the present application do not impose specific restrictions on this.

[0097] By obtaining sample query statements associated with the target knowledge domain and generating multiple sample meta-paths according to the entities and their relationships therein, it is possible to optimize the generation of query results through multiple sample meta-paths in the future.

[0098] Step 202, matching the entities contained in the domain knowledge graph and the association relationships between the entities according to the multiple sample meta-paths to obtain multiple sample candidate paths.

[0099] Exemplarily, a reinforcement learning object agent can be constructed, and a state space, action space and reward mechanism can be established. The object agent can be guided by a policy network, make serialized decisions in the domain knowledge graph according to each sample meta-path to search for the target entity in multiple hops, and construct a sample candidate path based on the sample first entity obtained by the decision at each time step.

[0100] Furthermore, the policy network can be a long short-term memory network (LSTM), which is used to capture the historical information of the object agent's actions in the domain knowledge graph and assist the object agent in making decisions. Specifically, the policy network can take the embedding vector of the current entity and the embedding vector of the previous action as input, output the probability distribution of each possible action in the next time step, and make a decision based on the probability distribution through the object agent to obtain the first entity of the sample that needs to be selected in the next step.

[0101] Furthermore, after the object agent generates an updated sample first entity for the next time step, the object agent will take corresponding actions, at which time the path reward for the object agent to perform the action can be calculated to encourage the object agent to gradually make better decisions.

[0102] Furthermore, after each corresponding sample candidate path is generated, the parameter gradient of the policy network can be calculated, and the parameters of the policy network can be updated according to the parameter gradient, thereby improving the reliability of the subsequent output probability of the policy network.

[0103] Through the above process, the agent will perform multi-hop reasoning in the knowledge graph based on the policy network to generate multiple sample candidate paths. Each path represents a possible answer or information flow, including the sample head entity, the sample tail entity, and multiple sets of relationships between them.

[0104] Step 203: Determine the corresponding sample target path according to the similarity between the sample query statement and the plurality of sample candidate paths.

[0105] Furthermore, multiple sample candidate paths can be converted into description text. For example, when the sample candidate path is e 肺癌 →r 被治疗 →e 吉西他滨 →r 影响 →e hENT1 , it can be converted into a description text "One of the reasons why gemcitabine is an effective treatment for lung cancer is that it affects the expression of hENT1 to a certain extent". The multiple description texts corresponding to the multiple sample candidate paths are input into the pre-trained retrieval model together with the sample query sentence, and the retrieval model generates multiple path embedding vectors corresponding to the multiple description texts and the first embedding vector corresponding to the query sentence.

[0106] Furthermore, the retrieval model can be used to calculate the similarity between each path embedding vector and the first embedding vector one by one, and select the sample candidate path corresponding to the path embedding vector with the highest similarity as the sample target path. Finally, the retrieval model will output the first embedding vector and the second embedding vector of the description text corresponding to the sample target path.

[0107] Exemplarily, the retrieval model may be an encoder model, such as a bidirectional encoder representation from a transformer (BERT) model. The retrieval model may be obtained by performing loss minimization training based on a preset retrieval model according to a similarity loss. For example, the similarity loss may be calculated by cosine similarity, as follows:

[0108]

[0109] Among them, q represents the sample first embedding vector, and p represents the sample path embedding vector.

[0110] By selecting the sample candidate path with the greatest similarity to the sample query sentence as the sample target path, a sample target path can be selected that enables the large language model to more accurately identify the true intent represented by the sample query sentence, thereby improving the quality and accuracy of the answer.

[0111] Step 204 , aggregating the first embedding vector of the sample query statement and the second embedding vector of the description text corresponding to the sample target path to obtain a sample embedding vector.

[0112] For example, the retrieval model can be used to convert the sample query statement into a first embedding vector, and the description and text corresponding to the sample target path into a second embedding vector to represent the semantic information of the text in a high-dimensional space, so that the large language model can better understand and learn the meaning of the text, rather than just superficial vocabulary matching.

[0113] Furthermore, the first embedding vector and the second embedding vector can be concatenated through the retrieval model, and then aggregated through a multi-layer perceptron to obtain a sample embedding vector. The concatenation operation can be achieved by vector concatenating the first embedding vector and the second embedding vector. The sample embedding vector contains the common features of the query statement and the sample target path description, which helps the large language model to more comprehensively understand the context and intent of the query, so as to learn how to align the sample query statement with the corresponding answer, thereby obtaining more accurate and relevant results.

[0114] Step 205 , concatenate the sample query statement with the query result of the sample query statement in the domain knowledge graph of the target knowledge domain to obtain a sample indication statement.

[0115] In some embodiments, in order to further improve the accuracy of the generated predicted sample query results, the corresponding context information can be searched from the domain knowledge graph based on the sample query statement to obtain the query result, and the sample query statement can be spliced ​​with the query result to help the large prediction model better understand the background and intent of the query.

[0116] For example, for the sample query statement "In the treatment of lung cancer, which genes are related to gemcitabine resistance?", the sample query statement is queried through the corresponding domain knowledge graph, and the query result "Lung cancer can be treated with gemcitabine, and gemcitabine affects hENT1 expression" can be obtained, thereby obtaining the sample indication statement "Question: In the treatment of lung cancer, which genes are related to gemcitabine resistance?, Background knowledge: Lung cancer can be treated with gemcitabine, and gemcitabine affects hENT1 expression".

[0117] By introducing sample instruction sentences, we can ensure that the generated answers are based on reliable data, and achieve alignment between questions and answers, effectively reducing hallucination problems.

[0118] Step 206: Generate a predicted sample query result based on the sample embedding vector and the sample indication statement.

[0119] Exemplarily, the sample embedding vector provides deep semantic information of the query statement and the descriptive text of the sample target path, while the sample indication statement provides background knowledge and contextual information of the query. In this way, the sample embedding vector and the sample indication statement can be input into the large language model together. The combination of the two enables the large language model to understand the intent of the query more accurately and generate more accurate and detailed answers.

[0120] Step 207, obtaining the real sample query result, and performing difference minimization training on the pre-trained large language model according to the predicted sample query result and the real sample query result, to obtain the target large language model.

[0121] In some embodiments, in order to effectively measure the difference between the predicted sample query results generated by the large language model and the real sample query results, the cross-entropy loss can be calculated to evaluate the degree of match between the predicted sample query results generated by the large language model and the real sample query results (i.e., the real sample labels). The lower the cross-entropy loss, the more accurate the predicted sample query results predicted by the large language model.

[0122] For example, a sample query statement may be "Which genes are associated with gemcitabine resistance in lung cancer treatment?". A predicted sample query result may be "One of the reasons why gemcitabine is an effective treatment for lung cancer is that it affects the expression of hENT1 to a certain extent." A real sample query result may be "In lung cancer treatment, genes associated with gemcitabine resistance include hENT1, because gemcitabine exerts its therapeutic effect by affecting the expression of hENT1.".

[0123] Exemplarily, the calculation formula of the cross entropy loss L is as follows:

[0124] L=-Σ c∈C y c log(y c ′ )

[0125] Among them, y c Represents the real sample query result, exemplarily, y c It can be one-hot encoded, for the correct category y c is 1, the rest are 0; c ′ Represents the predicted sample query result, specifically the probability that the predicted sample query result is the cth category, where C is the total category set.

[0126] In some implementations, the pre-trained large language model can be fine-tuned by generating multiple different query statements in combination with the domain knowledge graph until the number of training reaches a preset number, such as 500 or 1,000 times, or the cross entropy loss is less than a preset threshold, or the preset training time is reached, etc., then the fine-tuning of the large language model can be stopped, so that the large language model can fully learn knowledge in different fields.

[0127] By calculating the cross entropy loss and performing difference minimization training on the pre-trained large language model, the parameters of the pre-trained large language model can be effectively optimized so that the predicted sample query results it generates are closer to the real sample query results. In this way, the generation quality of the target large language model can be improved and the stability and robustness of the target large language model when processing complex tasks can be ensured.

[0128] In some implementations, in order to ensure the accuracy and reliability of the constructed sample candidate paths, the corresponding sample first entity may be recursively matched by the entity type of the sample meta-path to improve the quality of the sample candidate paths obtained by the query. For example, step 202 may include:

[0129] (202.1) determining, according to entity types in the multiple sample meta-paths, multiple sample first entities of the multiple sample meta-paths from multiple entities included in the domain knowledge graph;

[0130] (202.2) For each sample meta-path, repeatedly match the next sample first entity according to the association relationship between the sample first entity and other entities to obtain an updated sample first entity, until the first entity type of the updated sample first entity is the last entity type in each corresponding sample meta-path;

[0131] (202.3) According to the multiple sample first entities and the multiple updated sample first entities corresponding to the multiple sample meta-paths, a corresponding multiple sample candidate paths are sequentially constructed.

[0132] The entity type can be the category or label to which the entity belongs in the knowledge graph. For example, in the medical field, the entity type can include "disease", "drug", "gene", etc.

[0133] Among them, the sample first entity can be a specific entity found in the domain knowledge graph during the path search process according to the first entity type searched according to the current sample meta-path. The sample first entity is the starting point for the path search of each sample meta-path.

[0134] The updated sample first entity may be the next sample entity that meets the conditions found during the path search process according to the association relationship between the current sample first entity and other sample entities. This process is performed step by step, and each time a new sample entity is matched according to the sample meta-path, the sample entity becomes the new updated sample first entity.

[0135] For example, sample metapath 1 is lung cancer → therapeutic drug → gemcitabine → affecting gene → gene. For sample metapath 1, take "lung cancer" as the first entity of the sample, and start from "lung cancer" to search for sample entities with a "therapeutic drug" relationship with it, and get gemcitabine, which is the first entity of the Western medicine sample; start from "gemcitabine", search for sample entities with a "affecting gene" relationship with it, and get hENT1, which is the updated first entity of the sample; since the entity type of "hENT1" is "gene", it indicates that the end of the sample metapath has been reached, so the search is stopped, and the sample candidate path 1 is obtained: e 肺癌 →r 治疗药物 →e 吉西他滨 →r 影响基因 →e hENT1 .

[0136] Specifically, the object proxy agent can be used to search and make decisions in the domain knowledge graph according to the sample meta-path, and gradually generate sample candidate paths.

[0137] Similarly, other sample candidate paths can also be obtained according to the above method, and finally multiple sample candidate paths are obtained. The specific method is not described here.

[0138] Through the above steps, multiple sample candidate paths can be matched from the domain knowledge graph. The sample candidate paths not only meet the definition of sample meta-paths, but also ensure the accuracy and reliability of the paths.

[0139] In some implementations, since exhaustive methods or greedy algorithms have high computational complexity, low search efficiency, and are prone to falling into the global optimal solution, in order to efficiently and accurately determine the sample candidate paths that meet the query conditions, the object proxy agent can dynamically adjust the search strategy based on historical information and current status to optimize the path globally and ensure the overall quality of the path. For example, "matching the next sample first entity according to the association relationship between the sample first entity and other entities to obtain an updated sample first entity" in (202.2) may include:

[0140] (202.2.1) controlling the object agent to obtain multiple candidate sample entities according to the association relationship between the sample first entity and other entities;

[0141] (202.2.2) Output the predicted probability of each candidate sample entity being the first entity of the next sample through the policy network of the object agent;

[0142] (202.2.3) Based on multiple prediction probabilities of multiple candidate sample entities, the control object agent agent determines the updated sample first entity from the multiple candidate prediction probabilities with the goal of maximizing the path matching reward.

[0143] The object agent can be an intelligent agent that makes decisions during the path search process. The object agent can evaluate the predicted probability of each possible candidate sample entity (i.e., action) through the policy network, and select the updated sample first entity (i.e., the optimal action) based on historical information and the current state space.

[0144] The candidate sample entity may be a group of possible next sample first entities found during the path search process according to the association relationship between the current sample first entity and other entities. The candidate sample entity is a candidate object for the next step selection.

[0145] The predicted probability can be the probability of each candidate sample entity output by the policy network of the object agent becoming the first entity of the next sample. The predicted probability is used to assist in guiding the next choice of the object agent.

[0146] In some embodiments, the state space of the object proxy agent can be defined first, including the current node position (updated sample first entity), the sample starting entity (sample first entity), and the sample target entity (currently only the entity type is known); and the action space A of the object proxy agent can be defined: moving to a neighboring node (candidate sample entity) and stopping.

[0147] Furthermore, taking the sample meta-path of lung cancer → therapeutic drug → gemcitabine → influencing gene → gene as an example, "lung cancer" is found as the first sample entity from the domain knowledge graph. For sample meta-path 1, starting from "lung cancer", sample entities with a "therapeutic drug" relationship are searched, and for example, therapeutic drug a1 and therapeutic drug a2 are obtained, then therapeutic drug a1 and therapeutic drug a2 are selected as candidate sample entities.

[0148] Furthermore, the predicted probability of each candidate sample entity being the first entity of the next sample can be evaluated through the policy network of the object proxy agent. If the predicted probability of therapeutic drug a1 is 0.8 and the predicted probability of therapeutic drug a2 is 0.2, the object proxy agent can maximize the path matching reward and select the candidate sample entity with the largest predicted probability as the updated first sample entity, that is, select therapeutic drug a1 as the updated first sample entity.

[0149] In some embodiments, the predicted probability and the expected reward for each candidate sample entity as the next sample first entity according to the reward function can also be combined, and by combining the predicted probability and the estimated path matching reward, a candidate sample entity with the highest comprehensive score is selected as the updated sample first entity. For example, when therapeutic drug a1 is selected as the next sample first entity, the estimated path matching reward is 0.9, and when therapeutic drug a2 is selected as the next sample first entity, the estimated path matching reward is 0.7, then the comprehensive score of therapeutic drug a1 can be calculated as 0.8*0.9=0.72, and the comprehensive score of therapeutic drug a2 can be calculated as 0.2*0.7=0.14. The object agent can select the therapeutic drug a1 with the highest comprehensive score as the next sample entity. If the therapeutic drug a1 is gemcitabine, then gemcitabine is the updated first sample first entity.

[0150] It should be noted that the subsequent updated sample first entity can also be determined according to the above method. For details, please refer to the above and will not be described in detail here.

[0151] Through the object proxy agent, combined with a policy network with adjustable parameters, with the goal of maximizing the path matching reward, it is possible to ensure that the optimal path is efficiently found in the domain knowledge graph. This not only improves the accuracy and efficiency of path selection, but also improves the robustness and adaptability of the object proxy agent in multi-hop reasoning.

[0152] In some embodiments, in order to improve the quality of the object agent's decision-making, the historical information of the object agent's actions in the domain knowledge graph can be captured through the policy network, so that the object agent can select the optimal action for the next step based on the current state and historical information, thereby improving the accuracy of the object agent's decision-making and preventing the object agent from falling into a local optimum. For example, (202.2.2) may include:

[0153] (202.2.2.1) obtaining, through the policy network of the object proxy agent, a third embedding vector of the current sample first entity and a fourth embedding vector of the sample first entity at a time step before the current sample first entity;

[0154] (202.2.2.2) The hidden state of the policy network is updated according to the third embedding vector and the fourth embedding vector, and the predicted probability of each candidate sample entity being the first entity of the next sample is output according to the updated hidden state.

[0155] Among them, the policy network can be a model used to guide the object agent to make serialized decisions in the domain knowledge graph. For example, the policy network can be an LSTM model, and the policy network can combine the feedback of the reward function to continuously optimize its internal parameters to make it more efficient and accurate in future decisions.

[0156] The third embedding vector may be an embedding representation of the first entity of the currently input sample.

[0157] The fourth embedding vector may be an embedding representation of the first entity of the currently input sample at the previous time step.

[0158] The hidden state can be a state variable inside the policy network, which is used to store and transmit historical information in the sequence data. The hidden state is updated at each step, representing the path information from the initial sample first entity to the current first sample first entity, helping the object agent to make more intelligent decisions.

[0159] Specifically, the third embedding vector e t and the fourth embedding vector e t-1 As input to the policy network, the initial hidden state h t-1 and the initial cell state c t-1 It can be a zero vector or a specific initial state. Afterwards, the policy network can update the cell state and hidden state according to the third embedding vector and the fourth embedding vector. The policy network will update the updated hidden state h tThe input is to the fully connected layer, and the output of the fully connected layer is converted into the predicted probability of each candidate sample entity being the first entity of the next sample through the softmax function. It can be understood that when determining the next sample first entity of each sample first node in the sample candidate path, the predicted probability of each candidate sample entity being the first entity of the next sample can be output in the above manner, and the corresponding selection operation can be performed, which is not described in detail in the embodiments of the present application.

[0160] For example, if the sample query is "Which genes are associated with gemcitabine resistance in lung cancer treatment?", then if the current sample first entity is lung cancer, lung cancer is converted into the third embedding vector e t , in the initial state, the fourth embedding vector of the sample first entity of the previous time step of the current sample first entity can be a zero vector or a specific starting vector e t-1 , if the initial hidden state of the policy network at the migration time step is h t-1 , the initial cell state is c t-1 , the updated hidden state and cell state can be obtained through the third embedding vector and the fourth embedding vector, and the updated hidden state and cell state are input into the fully connected layer. The output of the fully connected layer is converted into the predicted probability of each candidate sample entity being the first entity of the next sample through the softmax function.

[0161] Furthermore, when the next sample first entity is selected, that is, the updated sample first entity is gene A, the above steps can be repeated starting from gene A until the termination condition is reached, such as reaching the last entity type of the sample meta-path.

[0162] In this way, the object agent can combine the information of the current node and the node of the previous time step, update the hidden state and output the predicted probability of each candidate sample entity, so as to make a more intelligent decision. In this way, not only the accuracy of path selection is improved, but also the robustness and adaptability of the object agent in multi-hop reasoning are enhanced.

[0163] In some embodiments, in order to ensure that the object proxy agent can efficiently and accurately find multiple sample candidate paths related to a given sample query statement in the domain knowledge graph, the path matching reward after the object proxy agent performs an action (that is, after reaching the updated sample first entity) can be determined, so as to provide timely feedback after each action of the object proxy agent to quantify the degree to which the action of the object proxy agent at each time step achieves its final goal, and motivate the object proxy agent to make correct decisions. For example, after (202.2.3), that is, based on multiple predicted probabilities of multiple candidate sample entities, the object proxy agent is controlled to maximize the path matching reward, and after determining the updated sample first entity from multiple candidate predicted probabilities, it also includes:

[0164] (202.2.3.1) after the object proxy agent reaches the updated sample first entity, comparing the first entity type of the updated sample first entity with the indicated entity type in the corresponding sample meta-path, and determining a first reward index for the object proxy agent based on the comparison result;

[0165] (202.2.3.2) Obtaining a cumulative path confidence score from the first sample first entity to the updated sample first entity generated according to the sample meta-path, and a path match between the current path constructed from the first sample first entity to the updated sample first entity and the corresponding indicated entity type of the sample meta-path;

[0166] (202.2.3.3) Obtaining a first reward amplitude parameter and a second reward amplitude parameter, and obtaining a second reward index according to the product of the first reward amplitude parameter, the second reward amplitude parameter, the path confidence score, and the path matching degree;

[0167] (202.2.3.4) Determine the path matching reward for the object agent to reach the updated sample first entity based on the sum of the first reward index and the second reward index.

[0168] The first reward index may be a reward score obtained when the object proxy agent follows the sample meta-path to reach the updated sample first entity and compares it with the correct entity type. When the object proxy agent successfully reaches the correct entity type represented by the corresponding meta-path along the sample meta-path, the first reward index is 1; otherwise, it is 0. In this way, it can be ensured as much as possible that the object proxy agent follows the pre-set path pattern, thereby increasing the possibility of finding the correct entity.

[0169] The path confidence score can be a quantitative assessment of the overall reliability of the currently generated intermediate path, where the intermediate path is a path from the first sample first entity generated according to the sample meta-path to the updated sample first entity. The path confidence score can be a value obtained by combining the confidences of the various relationships (edges) on the intermediate path in some form (such as taking the average or other statistical methods). A high path confidence score means that this intermediate path is more likely to lead to the correct entity.

[0170] The path matching degree can be used to measure the consistency between the current intermediate path and the expected sample meta-path. The higher the path matching degree, the more the currently generated intermediate path conforms to the preset sample meta-path.

[0171] The first reward amplitude parameter may be a hyperparameter, which may be used to adjust the influence of the path confidence score and the path matching degree on the path matching reward. The first reward amplitude parameter may be set by the user to adapt to different application scenarios and requirements.

[0172] The second reward amplitude parameter may be a hyperparameter, which may be used to adjust the impact of the path confidence score and the path matching degree on the path matching reward. Similarly, this parameter may be adjusted by the user according to the specific situation to achieve better performance.

[0173] Among them, the second reward index can be the product of the first reward amplitude parameter, the second reward amplitude parameter, the path confidence score and the path matching degree.

[0174] The path matching reward can be a comprehensive evaluation of the first reward index and the second reward index, which combines the two (usually by simple addition) to form a reward value that fully reflects the rationality and effectiveness of the current path selection of the object agent. The higher the path matching reward, the more likely the object agent is to move along a high-quality and expected path, and thus the more likely it is to find the correct answer.

[0175] In some implementations, various indicators can be combined, and each time the object agent reaches the updated sample first entity, that is, after taking action at the current time step, the total path matching reward that the current object agent should obtain in the current state can be calculated, so as to continuously guide the object agent to learn and optimize its behavior, so that the object agent can accurately complete the multi-hop path finding task. Exemplarily, the calculation formula of the reward function is as follows:

[0176]

[0177] in, is the calculation function of the first reward index. When the object agent reaches the correct type of entity according to the sample meta-path, the first reward index is 1, otherwise it is 0; v(P j ) is the path confidence score of the current sample meta-path, is the indicator function of path matching degree. If the current intermediate path matches the sample meta-path, the path matching degree is 1, otherwise it is 0. b is the first reward amplitude parameter, ω is the second reward amplitude parameter, The second reward index.

[0178] Exemplarily, the reward is combined with the calculation formula of the reward function to introduce the process of path matching reward. For example, if the sample query statement is "In the treatment of lung cancer, which genes are related to gemcitabine resistance?", if the first sample first entity determined is lung cancer, the sample meta-path is lung cancer→gene→gemcitabine resistance. If the updated sample first entity selected by the object agent is gene a, the type of gene a is compared with the entity type "gene" indicated in the sample meta-path. If "gene A" is indeed a gene type, the first reward index is 1. Otherwise, the first reward index is 0. Assuming that "gene A" is indeed a gene type, the first reward index is 1.

[0179] Furthermore, it can be determined that the path from the first sample first entity to the updated sample first entity is the intermediate path p. Then, when calculating the cumulative path confidence score from the first sample first entity to the updated sample first entity, if the intermediate path p is lung cancer → gene, the relationship on the intermediate path p can be converted into an embedding vector e r1 , averaging the embedding vectors of all relations on the intermediate path, we can get the embedding vector e of the intermediate path p .

[0180] Furthermore, the embedding vector e of the intermediate path can be calculated p The similarity with the embedding vectors of the first sample entity type (i.e., the head entity) and the last sample entity type (i.e., the target entity) of the sample meta-path is normalized to obtain the path confidence score. Specifically, the similarity calculation formula is as follows:

[0181] similarity(e p , e 肺癌 , e 基因A )=cos(e p , e 肺癌 )+cos(e p , e 基因A )

[0182] The calculation formula for the path confidence score is as follows:

[0183] Conf(P)=similarity(e p , e 肺癌 , e 基因A ) / max(similarity), fake design

[0184] It is calculated that Conf(P)=0.8.

[0185] Furthermore, since the current path (ie, the intermediate path p) constructed from the first sample first entity to the updated sample first entity is lung cancer→gene A, it matches the corresponding part in the sample meta-path lung cancer→gene→gemcitabine resistance, and therefore, the path matching degree is 1.

[0186] Further, assuming that the first reward amplitude parameter b preset by the technician is 1 and the second reward amplitude parameter ω is 0.5, then according to the product of the first reward amplitude parameter, the second reward amplitude parameter, the path confidence score and the path matching degree calculated above, the second reward index can be obtained as 0.4.

[0187] Furthermore, according to the sum of the first reward index and the second reward index calculated above, it can be obtained that the path matching reward is 1.4.

[0188] In some embodiments, each time the object agent performs an action (selects an updated sample first entity), a path matching reward can be calculated in the above manner, so that the object agent can obtain feedback in a timely manner, adjust measurements, and ensure that the generated sample candidate paths are both scientific and highly credible, thereby greatly improving the learning efficiency of the object agent and the quality of path selection.

[0189] In some embodiments, when the object proxy agent performs a series of actions in the environment (domain knowledge graph), after each complete sample candidate path is obtained, the comprehensive gradient parameters can be calculated for the newly generated sample candidate path, and the parameters of the policy network can be updated accordingly, so that the policy network gradually learns to select those paths that can bring more rewards, thereby achieving more efficient multi-hop reasoning. For example, after (202.3), that is, after sequentially constructing the corresponding multiple sample candidate paths according to the multiple sample first entities and the multiple updated sample first entities corresponding to the multiple sample meta-paths, it can also include:

[0190] (202.3.1) Obtain the path matching reward obtained after determining the first entity of each sample when the object agent generates each sample candidate path, as well as the gradient contribution to the policy parameters of the policy network;

[0191] (202.3.2) Obtain the first gradient parameter by multiplying each path matching reward by the corresponding gradient contribution;

[0192] (202.3.3) adding a plurality of first gradient parameters corresponding to a plurality of sample first entities to obtain a comprehensive gradient parameter;

[0193] (202.3.4) Obtain the expected value of the comprehensive gradient parameter, and update the parameters of the policy network according to the expected value to obtain an updated policy network.

[0194] Among them, the gradient contribution can be the degree of influence of an action on the update of policy network parameters after the agent takes an action. For example, the derivative of the path matching reward brought by the action generated by the current policy (that is, the action of selecting the first entity of the updated sample) on the policy parameters can be used. In short, the gradient contribution can be used to measure the contribution of the action of selecting the first entity of the updated sample to the optimization of the policy network parameters.

[0195] Among them, the first gradient parameter can be the result of multiplying the path matching reward of each sample candidate path by the gradient contribution of the path to the policy network parameters. This means that for each sample candidate path generated by the object agent, we pay attention to both the reward value of the path itself (i.e., the quality of the path) and how the path affects the learning process of the policy network. The first gradient parameter can be a quantitative indicator of the importance of the sample candidate path to the optimization of the policy network.

[0196] The comprehensive gradient parameter can be the sum of the first gradient parameters of multiple sample candidate paths. By accumulating the first gradient parameters of multiple samples, a more comprehensive comprehensive gradient parameter that reflects the overall trend can be obtained, which helps to improve the efficiency and accuracy of the policy network update.

[0197] The expected value can be the average value of the comprehensive gradient parameter, or the average degree of influence of each action on the optimization of the policy network parameters in multiple sample first entities in a given candidate sample path. By calculating the expected value of the comprehensive gradient parameter, the gradient update process can be further smoothed, and the random fluctuations caused by a single action can be reduced, making the update of the policy network more stable and reliable.

[0198] For example, when the object agent generates a complete sample candidate path, a certain reward can be given according to the actions taken by selecting multiple sample first entities. For each action (such as the path generation from sample first entity a to sample first entity b), it is necessary to calculate the gradient contribution of this action to the policy network parameter θ. Specifically, the partial derivative of the policy with respect to θ is calculated.

[0199] Furthermore, the path matching reward of each attempt can be multiplied by the gradient contribution of the corresponding action to obtain the first gradient parameter to determine the contribution of each action to the optimization of the policy network parameters.

[0200] Furthermore, when generating sample candidate paths, the first gradient parameters corresponding to multiple time steps (each time step corresponds to generating an action) can be added together to summarize the overall contribution of each action to the optimization of the policy network parameters and obtain a comprehensive gradient parameter.

[0201] Furthermore, the expected value of the comprehensive gradient parameter can be calculated, that is, the average value of the contribution of the actions of the first entity of each selected sample to the optimization of the policy network parameters. Based on this expected value, the parameters θ of the policy network can be adjusted to make better decisions in the future. The specific update formula can be based on the basic form of the REINFORCE algorithm, that is:

[0202]

[0203] Among them, R(S T+1 ) is the path matching reward, g θ～at is the gradient contribution, g θ～at R(S T+1 ) is the first gradient parameter, is the comprehensive gradient parameter, g θ J(θ) is the expected value of the comprehensive gradient parameter.

[0204] Through the above method, the policy network can be adjusted to tend to choose paths that can bring higher rewards. Based on this, the object agent will be able to efficiently select the optimal sample candidate path in a given domain knowledge graph.

[0205] Please refer to Figure 5 In some embodiments, in combination Figure 5 The overall technical solution of the present application is introduced. Exemplarily, after obtaining the sample query statement, multiple sample meta-paths can be generated according to the sample association relationship between each sample entity in the sample query statement. And through the object proxy agent, based on the multiple sample meta-paths, traverse the corresponding domain knowledge graph to obtain multiple sample candidate paths corresponding to each sample meta-path.

[0206] Furthermore, by generating embedding vectors of query statements and multiple sample candidate paths, and calculating the similarity between the first embedding vector of the query statement and multiple embedding vectors of the description texts corresponding to the multiple sample candidate paths, the sample candidate path corresponding to the embedding vector with the highest similarity to the first embedding vector can be determined as the sample target path. The first embedding vector is concatenated with the second embedding vector corresponding to the sample target path to obtain the sample embedding vector, thereby completing the alignment between the sample query statement and the sample target path closest to the correct answer, and effectively solving the hallucination problem of the large language model.

[0207] Furthermore, the domain knowledge graph can be searched based on the sample query statement to obtain the corresponding query result, and the sample indication statement can be obtained by concatenating the sample query statement and the query result.

[0208] Furthermore, the sample embedding vector and the sample instruction sentence can be input into the pre-trained large language model together, and the parameters of the pre-trained large language model can be fine-tuned, that is, difference minimization training can be performed, so that the pre-trained large language model can output more accurate query results while fully learning the domain knowledge graph.

[0209] The embodiment of the present application obtains the query sentence associated with the target knowledge domain; inputs the query sentence into the target large language model to obtain the knowledge query result; wherein, the target large language model is obtained by minimizing the difference training based on the predicted sample query result and the real sample query result by the pre-trained large language model, and the predicted sample query result is generated by the pre-trained large language model based on the sample embedding vector and the sample indication sentence, the sample embedding vector is obtained by aggregating the first embedding vector of the sample query sentence and the second embedding vector of the description text corresponding to the sample target path, and the sample indication sentence is obtained by splicing the query result of the sample query sentence and the sample query sentence in the domain knowledge graph of the target knowledge domain; wherein, the sample target path is determined according to the similarity between the sample query sentence and multiple sample candidate paths; each sample candidate path is matched from the entities contained in the domain knowledge graph and the association relationship between the entities according to multiple sample meta-paths, and multiple sample meta-paths are generated according to the sample association relationship between each sample entity in the sample query sentence. In this way, by predefining multiple sample meta-paths, the real sample candidate path can be retrieved in the domain knowledge graph, invalid search paths can be quickly excluded, computing power can be saved, and the influence of noise can be effectively avoided. At the same time, by calculating the similarity, the sample target path with the highest similarity to the sample query statement is determined, the question and the answer can be aligned, and a path search result that highly matches the user's intention can be generated. In addition, by inputting the sample embedding vector and the sample indication statement into the pre-trained large language model for fine-tuning, additional clues can be provided to the pre-trained large language model to help it better locate the core of the problem, while significantly improving the learning effect of the large language model in the target knowledge field and the accuracy of knowledge matching, thereby preventing the trained target large language model from answering query statements in the target knowledge field. Hallucination problems. In summary, the present application can improve the accuracy of query results generated by a trained target large language model by performing efficient and accurate knowledge enhancement training on the pre-trained large language model.

[0210] See also Figure 6 The embodiment of the present application further provides a knowledge query device based on a large language model, which can implement the above-mentioned knowledge query method based on a large language model. The knowledge query device based on a large language model includes:

[0211] An acquisition module 61 is used to acquire a query statement associated with a target knowledge domain;

[0212] The input module 62 is used to input the query statement into the target large language model to obtain the knowledge query result; wherein the target large language model is obtained by performing difference minimization training based on the predicted sample query result and the real sample query result by the pre-trained large language model, and the predicted sample query result is generated by the pre-trained large language model based on the sample embedding vector and the sample indication statement, the sample embedding vector is obtained by aggregating the first embedding vector of the sample query statement and the second embedding vector of the description text corresponding to the sample target path, and the sample indication statement is obtained by splicing the query result of the sample query statement and the sample query statement in the domain knowledge graph of the target knowledge domain; wherein the sample target path is determined based on the similarity between the sample query statement and multiple sample candidate paths; each sample candidate path is matched from the entities contained in the domain knowledge graph and the association relationship between the entities according to multiple sample meta-paths, and the multiple sample meta-paths are generated based on the sample association relationship between each sample entity in the sample query statement.

[0213] The specific implementation of the knowledge query device based on the large language model is basically the same as the specific implementation of the knowledge query method based on the large language model described above, and will not be repeated here. On the premise of meeting the requirements of the embodiments of the present application, the knowledge query device based on the large language model can also be provided with other functional modules to implement the knowledge query method based on the large language model in the above embodiments.

[0214] The embodiment of the present application also provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned knowledge query method based on a large language model when executing the computer program. The computer device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0215] See also Figure 7 , Figure 7 The hardware structure of a computer device according to another embodiment is shown, and the computer device includes:

[0216] The processor 71 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0217] The memory 72 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 72 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 72, and the processor 71 calls and executes the knowledge query method based on a large language model in the embodiment of this application;

[0218] Input / output interface 73, used to implement information input and output;

[0219] Communication interface 74, used to realize communication interaction between the device and other devices, which can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0220] A bus 75 that transmits information between the various components of the device (e.g., the processor 71, the memory 72, the input / output interface 73, and the communication interface 74);

[0221] The processor 71 , the memory 72 , the input / output interface 73 and the communication interface 74 are connected to each other in communication within the device via a bus 75 .

[0222] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned knowledge query method based on a large language model.

[0223] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0224] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0225] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0226] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0227] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0228] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0229] It should be understood that in the present application, "at least one (item)" and "several" refer to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0230] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0231] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0232] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0233] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0234] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A knowledge query method based on a large language model, characterized in that: The method comprises: Obtain the query statements associated with the target knowledge domain; Inputting the query sentence into the target large language model to obtain a knowledge query result; The target large language model is obtained by performing difference minimization training based on a pre-trained large language model and predicted sample query results and real sample query results. The predicted sample query results are generated by the pre-trained large language model based on a sample embedding vector and a sample indication sentence. The sample embedding vector is obtained by aggregating a first embedding vector of a sample sentence to be queried and a second embedding vector of a description text corresponding to a sample target path. The sample indication sentence is obtained by concatenating the sample sentence to be queried with a query result of the sample sentence to be queried in a domain knowledge graph of the target knowledge domain. Among them, the sample target path is determined according to the similarity between the sample query statement and multiple sample candidate paths; each sample candidate path is matched from the entities contained in the domain knowledge graph and the association relationships between the entities according to multiple sample meta-paths, and the multiple sample meta-paths are generated according to the sample association relationships between the sample entities in the sample query statement.

2. The knowledge query method based on a large language model according to claim 1, characterized in that: The target large language model is trained in the following way: Acquire sample query statements associated with the target knowledge domain, and generate corresponding multiple sample meta-paths according to sample association relationships between sample entities in the query statements; According to the multiple sample meta-paths, matching is performed from entities included in the domain knowledge graph and association relationships between the entities to obtain multiple sample candidate paths; Determine a corresponding sample target path according to the similarity between the sample query statement and multiple sample candidate paths; Aggregating the first embedding vector of the sample query sentence and the second embedding vector of the description text corresponding to the sample target path to obtain a sample embedding vector; According to the sample query statement and the query result of the sample query statement in the domain knowledge graph of the target knowledge domain, a sample indication statement is obtained; Generate a predicted sample query result based on the sample embedding vector and the sample indication sentence; A real sample query result is obtained, and according to the predicted sample query result and the real sample query result, a difference minimization training is performed on the pre-trained large language model to obtain a target large language model.

3. The knowledge query method based on a large language model according to claim 2 is characterized in that: The matching of entities contained in the domain knowledge graph and the association relationships between the entities according to the multiple sample meta-paths to obtain multiple sample candidate paths includes: According to entity types in the multiple sample meta-paths, determine multiple sample first entities of the multiple sample meta-paths from multiple entities included in the domain knowledge graph; For each sample meta-path, repeatedly matching the next sample first entity according to the association relationship between the sample first entity and other entities to obtain an updated sample first entity, until the first entity type of the updated sample first entity is the last entity type in the corresponding each sample meta-path; According to the plurality of sample first entities and the plurality of updated sample first entities corresponding to the plurality of sample meta-paths, a plurality of corresponding sample candidate paths are sequentially constructed.

4. The knowledge query method based on a large language model according to claim 3 is characterized in that: The matching of the next sample first entity according to the association relationship between the sample first entity and other entities to obtain an updated sample first entity includes: The control object agent obtains a plurality of candidate sample entities according to the association relationship between the sample first entity and other entities; Outputting the predicted probability of each candidate sample entity being the first entity of the next sample through the policy network of the object agent intelligent body; Based on the multiple predicted probabilities of the multiple candidate sample entities, the object proxy agent is controlled to maximize the path matching reward, and an updated sample first entity is determined from the multiple candidate predicted probabilities.

5. The knowledge query method based on a large language model according to claim 4 is characterized in that: Outputting the predicted probability of each candidate sample entity being the first entity of the next sample through the policy network of the object proxy agent includes: Obtaining, through the policy network of the object proxy agent, a third embedding vector of the current sample first entity and a fourth embedding vector of the sample first entity at a previous time step of the current sample first entity; The hidden state of the policy network is updated according to the third embedding vector and the fourth embedding vector, and the predicted probability of each candidate sample entity being the first entity of the next sample is output according to the updated hidden state.

6. The knowledge query method based on a large language model according to claim 4, characterized in that: After the method controls the object proxy agent to maximize the path matching reward based on the multiple predicted probabilities of the multiple candidate sample entities and determines the updated sample first entity from the multiple candidate predicted probabilities, it also includes: After the object proxy agent reaches the updated sample first entity, the first entity type of the updated sample first entity is compared with the indicated entity type in the corresponding sample meta-path, and a first reward index for the object proxy agent is determined according to the comparison result; Obtaining a cumulative path confidence score from a first sample first entity generated according to the sample meta-path to the updated sample first entity, and a path matching degree between a current path constructed from the first sample first entity to the updated sample first entity and a corresponding indicated entity type of the sample meta-path; Obtaining a first reward amplitude parameter and a second reward amplitude parameter, and obtaining a second reward index according to the product of the first reward amplitude parameter, the second reward amplitude parameter, the path confidence score, and the path matching degree; According to the sum of the first reward index and the second reward index, a path matching reward for the object agent to reach the updated sample first entity is determined.

7. The knowledge query method based on a large language model according to claim 4, characterized in that: After sequentially constructing a plurality of corresponding sample candidate paths according to the plurality of sample first entities and the plurality of updated sample first entities corresponding to the plurality of sample meta-paths, the method further includes: Obtaining a path matching reward obtained after determining the first entity of each sample when the object proxy agent generates each sample candidate path, and a gradient contribution to the policy parameters of the policy network; Obtaining a first gradient parameter by multiplying each of the path matching rewards by the corresponding gradient contribution; Adding a plurality of first gradient parameters corresponding to the plurality of sample first entities to obtain a comprehensive gradient parameter; The expected value of the comprehensive gradient parameter is obtained, and the parameters of the policy network are updated according to the expected value to obtain an updated policy network.

8. A knowledge query device based on a large language model, characterized in that: The device comprises: An acquisition module is used to acquire the query statements associated with the target knowledge domain; An input module is used to input the query statement into a target large language model to obtain a knowledge query result; wherein the target large language model is obtained by performing difference minimization training based on a pre-trained large language model on the basis of predicted sample query results and real sample query results; the predicted sample query result is generated by the pre-trained large language model based on a sample embedding vector and a sample indication statement; the sample embedding vector is obtained by aggregating a first embedding vector of the sample query statement and a second embedding vector of a description text corresponding to a sample target path; the sample indication statement is obtained by splicing the sample query statement with a query result of the sample query statement in a domain knowledge graph of the target knowledge domain; wherein the sample target path is determined based on the similarity between the sample query statement and a plurality of sample candidate paths; each sample candidate path is obtained by matching the entities contained in the domain knowledge graph and the association relationships between the entities according to a plurality of sample meta-paths; the plurality of sample meta-paths are generated based on the sample association relationships between the sample entities in the sample query statement.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the knowledge query method based on a large language model as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the knowledge query method based on a large language model described in any one of claims 1 to 7 is implemented.

Images