Output detection method for large language model, apparatus, electronic device and storage medium

Abstract

The present invention provides an output detection method for a large language model, comprising: acquiring an output text of a large language model for a questioning text, and acquiring a target knowledge graph of a field to which the questioning text belongs, the target knowledge graph comprising knowledge data in the field to which the questioning text belongs; performing knowledge extraction on the output text to obtain knowledge data to be detected of the output text; and by means of the target knowledge graph, detecting the knowledge data to be detected, so as to obtain a knowledge detection result of the output text. Knowledge extraction is performed on the output text of the large language model so as to obtain the knowledge data to be detected of the output text, and the target knowledge graph in the field is used to detect the knowledge data to be detected, such that whether there is a knowledge error in the output text can be determined, thus preventing the large language model from outputting incorrect knowledge to users.

Description

Output detection method, device, electronic device and storage medium of large language model

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 30, 2023, with application number 202311870542.X and invention name “Output detection method, device, electronic device and storage medium of large language model”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present invention relates to the field of artificial intelligence, and in particular to a method, device, electronic device, and storage medium for detecting the output of a large language model. Background Art

[0003] In the rapid development of general artificial intelligence (AI), large language models have a head start. This is due to three factors. First, compared to other modalities (such as images, videos, and audio), text data is characterized by high knowledge content and data sparseness. Second, text data, leveraging the internet, has already digitized knowledge worldwide. Third, textual knowledge is organized by language and possesses a definite semantic structure, which can be represented and learned using deep learning models (such as the Transformer).

[0004] Based on the Transformer and massive amounts of textual knowledge, the large language model uses autoregressive modeling to achieve rich multilingual semantic learning. Autoregressive modeling has many advantages. One is that it can initiate parallel training based on the server; the other is that it can quickly learn the semantic information of the language. However, autoregressive modeling also brings the problem of hallucination. Specifically, the hallucination problem refers to the large language model outputting knowledge that does not exist in the training data, and this knowledge is incorrect. The difficulty of the hallucination problem lies in the difficulty of confirming whether the output knowledge exists in the training data through technical means, and it is also difficult to determine whether the knowledge is accurate, resulting in knowledge errors in the content output by the large language model.

[0005] Summary of the Invention

[0006] An embodiment of the present invention provides a method for detecting the output of a large language model, aiming to address the problem of knowledge errors in the output of large language models. By performing knowledge extraction on the output text of the large language model, knowledge data to be detected is obtained for the output text. This knowledge data is then detected using the target knowledge graph of the corresponding domain. This method can determine whether the output text contains knowledge errors, thus preventing the large language model from outputting incorrect knowledge to the user.

[0007] In a first aspect, an embodiment of the present invention provides an output detection method for a large language model, the method comprising:

[0008] Obtaining an output text of the large language model for the question text, and obtaining a target knowledge graph of the field to which the question text belongs, wherein the target knowledge graph includes knowledge data of the field to which the question text belongs;

[0009] Performing knowledge extraction on the output text to obtain knowledge data to be detected of the output text;

[0010] The knowledge data to be detected is detected through the target knowledge graph to obtain the knowledge detection result of the output text.

[0011] Optionally, before obtaining the target knowledge graph of the field to which the question text belongs, the method further includes:

[0012] Obtain entity tokens and relationship tokens from different fields;

[0013] For the entity words and the relationship words in a field, different entity words are annotated by the relationship words to obtain knowledge data, wherein the knowledge data includes at least two entity words and at least one relationship word associated with at least two entity words;

[0014] Construct knowledge graphs in different fields based on the knowledge data in different fields.

[0015] Optionally, the step of labeling different entity tokens using the relationship tokens to obtain knowledge data includes:

[0016] Dividing the entity word into subject word and object word;

[0017] The subject word-gram and the object word-gram are labeled based on the relation word-gram to obtain knowledge data.

[0018] Optionally, before performing knowledge extraction on the output text to obtain the knowledge data to be detected of the output text, the method further includes:

[0019] Obtaining a natural language processing model to be trained and a training data set, wherein the training data set includes sample text and knowledge labels of the sample text, wherein the input of the natural language processing model to be trained is constructed as the sample text, and the output of the natural language processing model to be trained is constructed as predicted knowledge data;

[0020] Inputting the sample text into the natural language processing model to obtain predicted knowledge data of the sample text;

[0021] Calculating a loss value between the predicted knowledge data of the sample text and the knowledge label of the sample text;

[0022] The parameters of the natural language processing model to be trained are adjusted based on the loss value, and the parameter adjustment process is iterated. After the training is completed, a trained natural language processing model is obtained, and the trained natural language processing model is used to extract knowledge from the output text.

[0023] Optionally, before performing knowledge extraction on the output text to obtain the knowledge data to be detected of the output text, the method further includes:

[0024] Obtaining a natural language processing model to be trained and a training dataset, wherein the training dataset includes sample text and entity labels and relationship labels of the sample text, wherein the input of the natural language processing model to be trained is constructed as the sample text, and the output of the natural language processing model to be trained is constructed as predicted entities and predicted relationships;

[0025] Inputting the sample text into the natural language processing model to obtain predicted entity data and predicted relationship data of the sample text;

[0026] Calculating a first loss value between the predicted entity data of the sample text and the entity label of the sample text, and a second loss value between the predicted relationship data of the sample text and the relationship label of the sample text;

[0027] Based on the first loss value and the second loss value, the parameters of the natural language processing model to be trained are adjusted, and the parameter adjustment process is iterated. After the training is completed, a trained natural language processing model is obtained, and the trained natural language processing model is used to extract knowledge from the output text.

[0028] Optionally, performing knowledge extraction on the output text to obtain knowledge data to be detected of the output text includes:

[0029] Performing knowledge extraction on the output text using the trained natural language processing model to obtain entities to be detected and relationships to be detected in the output text;

[0030] The entities to be detected and the relationships to be detected are combined to obtain knowledge data of the output text, where the knowledge data to be detected includes the entities to be detected and the relationships to be detected.

[0031] Optionally, the detecting the knowledge data to be detected by using the target knowledge graph to obtain the knowledge detection result of the output text includes:

[0032] Determine whether the entity to be detected and the relationship to be detected exist in the target knowledge graph;

[0033] If the entity to be detected exists in the target knowledge graph, and the relationship to be detected also exists in the target knowledge graph, then it is determined that the knowledge detection of the output text is correct;

[0034] If the entity to be detected does not exist in the target knowledge graph, and / or the relationship to be detected does not exist in the target knowledge graph, it is determined that the knowledge detection error of the output text is wrong.

[0035] In a second aspect, an embodiment of the present invention further provides an output detection device for a large language model, the output detection device for a large language model comprising:

[0036] A first acquisition module is configured to obtain an output text of the large language model for the question text, and to obtain a target knowledge graph of the field to which the question text belongs, wherein the target knowledge graph includes knowledge data of the field to which the question text belongs;

[0037] An extraction module, configured to perform knowledge extraction on the output text to obtain knowledge data to be detected in the output text;

[0038] The detection module is used to detect the knowledge data to be detected through the target knowledge graph to obtain the knowledge detection result of the output text.

[0039] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and runnable on the processor, wherein when the processor executes the computer program, the steps in the output detection method of the large language model provided in the embodiment of the present invention are implemented.

[0040] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the output detection method of the large language model provided in the embodiment of the invention are implemented.

[0041] In an embodiment of the present invention, the output text of a large language model for a question text is obtained, as is a target knowledge graph for the field to which the question text belongs, wherein the target knowledge graph includes knowledge data for the field to which the question text belongs; knowledge extraction is performed on the output text to obtain knowledge data to be tested for the output text; and the knowledge data to be tested is tested using the target knowledge graph to obtain a knowledge test result for the output text. By performing knowledge extraction on the output text of the large language model to obtain the knowledge data to be tested for the output text, and using the target knowledge graph for the field to which it belongs to test the knowledge data to be tested, it is possible to determine whether there are knowledge errors in the output text, thereby preventing the large language model from outputting erroneous knowledge to the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] FIG1 is a flow chart of an output detection method for a large language model provided by an embodiment of the present invention;

[0044] FIG2 is a flow chart of another large language model output detection method provided by an embodiment of the present invention;

[0045] FIG3 is a flow chart of a knowledge graph detection method provided by an embodiment of the present invention;

[0046] FIG4 is a flow chart of a secondary detection method provided by an embodiment of the present invention;

[0047] FIG5 is a schematic diagram of the structure of an output detection device for a large language model provided by an embodiment of the present invention;

[0048] FIG6 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0050] As shown in FIG1 , FIG1 is a flow chart of a method for detecting an output of a large language model provided by an embodiment of the present invention. The method for detecting an output of a large language model comprises the following steps:

[0051] 101. Obtain the output text of the large language model for the question text, and obtain the target knowledge graph of the field to which the question text belongs.

[0052] In an embodiment of the present invention, the large language model (LLM) can be a large language model obtained using an auto-regressive model, or any large language model that can produce an output illusion. The question text is the question text input by the user into the large language model to obtain the desired answer. The large language model performs semantic understanding and knowledge search based on the question text, and assembles the searched knowledge into natural language text for output to obtain output text. The target knowledge graph includes knowledge data in the field to which the question text belongs.

[0053] It should be noted that the large language model based on autoregressive modeling will inevitably have the problem of hallucination. In the process of implementing the large language model, users require the model to have certain innovation and accuracy. Innovation requires the large language model to process and understand the training data, and finally form some new knowledge, and this knowledge is likely to be obtained based on reasoning. Accuracy requires that the large language model cannot arbitrarily fabricate and synthesize based on the training data, resulting in the output of incorrect knowledge. Innovation and accuracy are mutually restrictive and require a balance. After the large language model is trained and learned based on massive text data, it often produces a certain degree of innovation, which brings about the problem of hallucination. For all correct knowledge, an abstract mathematical set K can be defined. The knowledge contained in massive text data is defined as X. Generally speaking, there can be text knowledge X≤K. Suppose a large language model f is trained based on X. Given a specific input x, the large language model has an output y=f(x). When The large model can be considered to be hallucinating. It's worth noting that as the boundaries of human knowledge expand, K is a dynamically growing set. Taking the set K at a given point in time, we can determine whether the large model's output y = f(x) is hallucinating. However, in practical applications, it's impossible to construct a domain-complete set K. Furthermore, making judgments based on X based on the output of a large model is extremely resource-intensive, as X often involves hundreds of billions of words, and the knowledge contained therein is difficult to retrieve. Therefore, actually detecting whether a large model's output is hallucinating is a difficult problem.

[0054] In this embodiment of the present invention, the knowledge graph is constructed by domain, avoiding the collection of cross-domain knowledge. Instead of considering the completeness of knowledge across all domains, only the completeness of knowledge in each domain needs to be considered. These domains could be secondary education, corporate finance, surgical medicine, and so on. It is understood that the finer the domain division, the easier it is to construct the knowledge graph, and the smaller the constructed knowledge graph, the easier it is to achieve 100% knowledge accuracy.

[0055] This knowledge data is encapsulated according to a pre-set knowledge structure, for example, by entities and relationships, specifically by relationships between entities. These entities and relationships can be different types of tokens. Entity tokens can be categorized as names, pronouns, organizations, locations, and time periods, while relationship tokens can be categorized as inheritance, synonymy, attribution, correlation, causality, and so on. Two entity tokens are combined using relationship tokens to create a piece of knowledge data. The knowledge data in each field's knowledge graph must be reviewed by professionals in the field to ensure 100% accuracy.

[0056] Different knowledge graphs correspond to different fields. The large language model can output the output text and the knowledge graph interface API of the question text. The knowledge graph interface API corresponds to the knowledge graph of the field to which the question text belongs. The corresponding knowledge graph can be called through the knowledge graph interface API. Different knowledge graphs correspond to different knowledge graph interface APIs. The target knowledge graph of the field to which the question text belongs can be loaded or called through the knowledge graph interface API. Of course, it is also possible to directly perform natural language processing based on the question text to determine the field to which the question text belongs, and load or call the target knowledge graph of the field to which the question text belongs based on the field to which the question text belongs, without the need to determine the field to which the question text belongs through a large language model. The embodiment of the present invention preferably outputs the output text and the knowledge graph interface API of the question text through a large language model. In this way, it is only necessary to add prompt words to the question text, for example, add "and output the knowledge graph interface API of the field corresponding to the question", and there is no need to train additional field classifiers.

[0057] 102. Perform knowledge extraction on the output text to obtain knowledge data to be detected of the output text.

[0058] In this embodiment of the present invention, the output text is processed and output by the large language model based on the user's question text. This output text is not output to the user as the final output text. Instead, it is subject to a knowledge check. If the knowledge check passes, the output text is output to the user as the final output text.

[0059] Entities and relations can be extracted from the output text through sequence labeling technology, and the knowledge data of the output text can be obtained as the knowledge data to be detected through the combination of entities and relations.

[0060] You can also train a knowledge data extraction model and use it to extract knowledge data from the output text to obtain the knowledge data to be tested. The structure of the knowledge data to be tested is the same as that of the knowledge data in the knowledge graph.

[0061] In an output text, the knowledge data to be detected can be one or more pieces.

[0062] 103. The knowledge data to be tested is tested through the target knowledge graph to obtain the knowledge detection result of the output text.

[0063] In an embodiment of the present invention, after obtaining the knowledge data to be tested, the knowledge data to be tested is matched with all the knowledge data in the target knowledge graph. If all the knowledge data to be tested matches the knowledge data in the knowledge graph, it indicates that there is no incorrect knowledge in the output text, and the next step is to determine that the knowledge test result of the output text is passed. If there is any knowledge data to be tested that is not matched with the knowledge data in the knowledge graph, it indicates that there is incorrect knowledge in the output text, and the next step is to determine that the knowledge test result of the output text is failed.

[0064] After obtaining the knowledge check result for the output text, the output text that passed the knowledge check can be used as the final output text and output to the user. If the output text fails the knowledge check, the large language model can be prompted to regenerate new output text based on the question text until the new output text passes the knowledge check, or until the number of knowledge check failures reaches a preset number, prompting the user to re-describe the question text.

[0065] In an embodiment of the present invention, the output text of a large language model for a question text is obtained, as is a target knowledge graph for the field to which the question text belongs, wherein the target knowledge graph includes knowledge data for the field to which the question text belongs; knowledge extraction is performed on the output text to obtain knowledge data to be tested for the output text; and the knowledge data to be tested is tested using the target knowledge graph to obtain a knowledge test result for the output text. By performing knowledge extraction on the output text of the large language model to obtain the knowledge data to be tested for the output text, and using the target knowledge graph for the field to which it belongs to test the knowledge data to be tested, it is possible to determine whether there are knowledge errors in the output text, thereby preventing the large language model from outputting erroneous knowledge to the user.

[0066] Optionally, before the step of obtaining the target knowledge graph of the field to which the question text belongs, entity terms and relationship terms from different fields can also be obtained; for the entity terms and relationship terms of a field, different entity terms are marked by relationship terms to obtain knowledge data, which includes at least two entity terms and at least one relationship term associated with at least two entity terms; knowledge graphs for different fields are constructed based on the knowledge data from different fields.

[0067] In the embodiment of the present invention, a knowledge graph is constructed for each field. Entity terms and relationship terms in each field can be collected and sorted, and the entity terms and relationship terms are associated according to the structure of the knowledge data to obtain the knowledge data.

[0068] Generally speaking, a knowledge graph is a node-relationship graph that represents knowledge through nodes and connections. Each node corresponds to an entity, and the relationships between entities are the connections between corresponding nodes. Entities consist of subjects and objects. Knowledge data in a knowledge graph is formed by entities and relationships. For example, if "cat" is an entity token, "animal" is an entity token, and the relationship token "belongs to" represents the knowledge that cat belongs to animals; if "grandma" is an entity token, "dad" is an entity token, and "mom" is an entity token, and the relationship token between "grandma" and "dad" is an entity token, and the relationship token between "dad" and "mom" is an entity token, then the knowledge represented is that "dad's mom is grandma." It is worth noting that due to the complex combinations of entities and relationships, the knowledge graphs in the embodiments of the present invention are domain- or scenario-specific. In practical applications, a knowledge graph is constructed for a specific scenario. For example, if the scenarios are secondary education, corporate finance, or surgical medicine, corresponding knowledge graphs are constructed for secondary education, corporate finance, or surgical medicine, respectively. Knowledge graphs can be constructed by manually annotating entities and relationships, or by generating a model and then manually verifying it. After the knowledge graph is constructed, its accuracy needs to be strictly checked. In order to achieve the desired application effect, the accuracy of the knowledge graph must be 100%.

[0069] Optionally, in the step of labeling different entity terms through relational terms to obtain knowledge data, the entity terms can be divided into subject terms and object terms; the subject terms and object terms are labeled based on the relational terms to obtain knowledge data.

[0070] In embodiments of the present invention, the subject token may represent the subject in a piece of knowledge data, and the object token may represent the object in a piece of knowledge data. A piece of knowledge data may be composed of subject tokens, object tokens, and relation tokens. For example, if cat is the subject, animal is the object, and the relation is belongs to, the knowledge represented is that cat belongs to animal.

[0071] Knowledge data can be obtained by manually labeling the subject and object terms using relational terms, or by generating knowledge data through a model and then manually verifying it to obtain verified knowledge data.

[0072] By dividing entity words into subject words and object words, knowledge data can be made clearer and more standardized, the matchability of the knowledge graph is improved, and the matching speed of knowledge data is increased.

[0073] Optionally, before performing knowledge extraction on the output text and obtaining the knowledge data to be detected of the output text, a natural language processing model to be trained and a training data set can also be obtained, the training data set including sample text and knowledge labels of the sample text, the input of the natural language processing model to be trained is constructed as sample text, and the output of the natural language processing model to be trained is constructed as predicted knowledge data; the sample text is input into the natural language processing model to obtain the predicted knowledge data of the sample text; the loss value between the predicted knowledge data of the sample text and the knowledge label of the sample text is calculated; the parameters of the natural language processing model to be trained are adjusted based on the loss value, and the parameter adjustment process is iterated. After the training is completed, a trained natural language processing model is obtained, and the trained natural language processing model is used to perform knowledge extraction on the output text.

[0074] In the embodiment of the present invention, knowledge extraction can be performed on the output text using a trained natural language processing model to obtain knowledge data to be detected in the output text.

[0075] Specifically, a natural language processing model to be trained and a training data set are constructed, and the natural language processing model to be trained is supervisedly trained using the training data set. After the training is completed, a trained natural language processing model is obtained.

[0076] The natural language processing model to be trained can be constructed based on RNN, LSTM, Transformer, etc., and can be, for example, a BERT or TF-IDF model. The input of the natural language processing model to be trained is constructed as text output, and the output of the natural language processing model to be trained is constructed as predicted knowledge data. That is, a text is input into the natural language processing model to be trained, and the output is predicted knowledge data. The number of predicted knowledge data can be one or more.

[0077] The training data set includes sample texts and knowledge labels of the sample texts. The knowledge labels may include one or more correct knowledge data, and the correct knowledge data is obtained based on manual annotation.

[0078] During the training process, a sample text can be input into the natural language processing model to be trained. The sample text is then processed by the natural language processing model to obtain predicted knowledge data for the sample text. A preset loss function is used to calculate the loss between the predicted knowledge data of the sample text and the knowledge label of the sample text. The model parameters of the natural language processing model to be trained are updated and adjusted with minimizing the loss as the optimization goal. This update and adjustment process is iterated until the number of iterations reaches a preset number or the loss converges to the minimum loss value, completing the training and obtaining a trained natural language processing model. This loss function can be a cross-entropy loss function, a mean squared error loss function, a logarithmic loss function, or the like.

[0079] After training is complete, the trained NLP model must be tested to determine its accuracy. The model that meets the accuracy requirements will be considered the final trained NLP model. For example, a trained NLP model must achieve a knowledge data extraction accuracy of at least 95% before it can be used in practical applications. If this accuracy requirement is not met, additional training data is added to improve the accuracy, ultimately resulting in a trained NLP model that can be put into practical use.

[0080] Optionally, before performing knowledge extraction on the output text and obtaining the knowledge data to be detected of the output text, a natural language processing model to be trained and a training data set may be obtained, the training data set including sample text and entity labels and relationship labels of the sample text. The input of the natural language processing model to be trained is constructed as sample text, and the output of the natural language processing model to be trained is constructed as predicted entities and predicted relationships. The sample text is input into the natural language processing model to obtain predicted entity data and predicted relationship data of the sample text. A first loss value between the predicted entity data of the sample text and the entity label of the sample text, and a second loss value between the predicted relationship data of the sample text and the relationship label of the sample text are calculated. The parameters of the natural language processing model to be trained are adjusted based on the first loss value and the second loss value, and the parameter adjustment process is iterated. After the training is completed, a trained natural language processing model is obtained, and the trained natural language processing model is used to perform knowledge extraction on the output text.

[0081] In the embodiment of the present invention, knowledge extraction can be performed on the output text using a trained natural language processing model to obtain knowledge data to be detected in the output text.

[0082] Specifically, a natural language processing model to be trained and a training data set are constructed, and the natural language processing model to be trained is supervisedly trained using the training data set. After the training is completed, a trained natural language processing model is obtained.

[0083] The natural language processing model to be trained can be constructed based on RNN, LSTM, Transformer, etc., and can be, for example, a BERT or TF-IDF model. The input of the natural language processing model to be trained is constructed as text output, and the output of the natural language processing model to be trained is constructed as predicted entity data and predicted relationship data. That is, a text is input to the natural language processing model to be trained, and the output is predicted entity data and predicted relationship data. The number of predicted entity data and predicted relationship data can be one or more.

[0084] The above training data set includes sample texts and entity labels and relationship labels of the sample texts, and the entity labels and relationship labels are obtained based on manual annotation.

[0085] During the training process, the sample text can be input into the natural language processing model to be trained, and the sample text can be processed by the natural language processing model to be trained to obtain the predicted entity data and predicted relationship data of the sample text. A first loss value between the predicted entity data of the sample text and the entity label of the sample text, and a second loss value between the predicted relationship data of the sample text and the relationship label of the sample text are calculated by a preset loss function. The first loss value and the second loss value are added to obtain a total loss value. The model parameters of the natural language processing model to be trained are updated and adjusted with minimizing the total loss value as the optimization goal. The above-mentioned update and adjustment process is iterated until the number of iterations reaches the preset number or the total loss value converges at the minimum loss value, and the training is completed to obtain a trained natural language processing model. The above-mentioned loss function can be a cross entropy loss function, a mean square error loss function, a logarithmic loss function, etc.

[0086] After training is complete, the trained NLP model must be tested to determine its accuracy. The model that meets the accuracy requirements will be considered the final trained NLP model. For example, a trained NLP model must achieve a knowledge data extraction accuracy of at least 95% before it can be used in practical applications. If this accuracy requirement is not met, additional training data is added to improve the accuracy, ultimately resulting in a trained NLP model that can be put into practical use.

[0087] Optionally, in the step of performing knowledge extraction on the output text to obtain the knowledge data to be detected of the output text, knowledge extraction can be performed on the output text through a trained natural language processing model to obtain the entities to be detected and the relationships to be detected of the output text; the entities to be detected and the relationships to be detected are combined to obtain the knowledge data of the output text, and the knowledge data to be detected includes the entities to be detected and the relationships to be detected.

[0088] In this embodiment of the present invention, the output text based on the large language model can be used by a trained natural language processing model to perform relationship and entity recognition. Entity recognition classifications include names, pronouns, organizations, locations, and time. Relationship recognition classifications include inheritance, synonymy, attribution, correlation, and causality.

[0089] In one embodiment, sequence labeling technology is used to extract entities, which are then identified and classified using BERT. Relationship classification is then performed directly based on the extracted entities and categorized using BERT. Based on a BERT model that meets the required accuracy, the output text of a large language model is obtained and used as the BERT input text. After obtaining the BERT output, all entity and relationship combinations in the output text can be generated.

[0090] Optionally, in the step of detecting the knowledge data to be detected through the target knowledge graph and obtaining the knowledge detection result of the output text, it can be determined whether the entity to be detected and the relationship to be detected exist in the target knowledge graph; if the entity to be detected exists in the target knowledge graph, and the relationship to be detected also exists in the target knowledge graph, then it is determined that the knowledge detection of the output text is correct; if the entity to be detected does not exist in the target knowledge graph, and / or the relationship to be detected does not exist in the target knowledge graph, then it is determined that the knowledge detection of the output text is wrong.

[0091] In an embodiment of the present invention, a secondary knowledge check of the knowledge graph can be implemented by node and relationship matching based on graph matching technology. Specifically, all entity and relationship combinations are input into the target graph. First, it is determined whether the entity to be detected exists in the target graph. If any entity to be detected does not exist in the target graph, it is determined that the knowledge detection error of the output text is wrong, and the knowledge detection error is returned. If all entities to be detected exist in the target graph, then the next step is to determine whether the relationship to be detected exists in the target graph. If any relationship to be detected does not exist in the target graph, then the knowledge detection error of the output text is determined, and the knowledge detection error is returned. If all relationships to be detected exist in the target graph, then the knowledge detection of the output text is determined to be correct, and the knowledge detection correct is returned.

[0092] Specifically, FIG2 is used for explanation. FIG2 is a flowchart of another large language model output detection method provided by an embodiment of the present invention. In FIG2, the method includes the following steps:

[0093] 201. Text input.

[0094] The above text input may be a question text of the user.

[0095] 202. Large Language Model (LLM).

[0096] The text input is processed through a large language model.

[0097] 203. Text output y.

[0098] The above text output y is the output text of the large language model.

[0099] 204. Knowledge graph detection.

[0100] The output text is subjected to knowledge detection through the target knowledge graph. If the detection fails, the process proceeds to step 205; if the detection passes, the process proceeds to step 206.

[0101] 205. Refuse to answer.

[0102] 206. Second confirmation output.

[0103] Further, FIG3 is used for illustration. FIG3 is a flow chart of a knowledge graph detection method provided by an embodiment of the present invention. FIG3 includes the following steps:

[0104] 301. Relationship and entity extraction based on BERT.

[0105] Based on Bert, the relations and entities in the text output y are extracted to obtain the entity-relationship pair set C of the text output.

[0106] 302. For any element c∈C, check whether c∈G is satisfied.

[0107] Check whether any element c in the set C is also in the target graph G.

[0108] 303. Return the knowledge graph detection result.

[0109] If any element c∈C satisfies c∈G, it can be determined that the detection is passed, otherwise it is determined that the detection is failed.

[0110] Furthermore, based on the Bert model that meets the accuracy standards, the output of the large language model is obtained, and the output of the large model is used as the Bert input. After obtaining the Bert output, all entity and relationship combinations can be formed. That is, the output is a series of "subject-relationship-object" knowledge points. Based on graph matching technology, secondary knowledge detection of the knowledge graph is achieved through node and relationship matching. This is illustrated in conjunction with Figure 4, which is a flow chart of a secondary detection method provided by an embodiment of the present invention. In Figure 4, the following steps are included:

[0111] 401. All entity and relationship combinations are input.

[0112] All entity and relationship combinations output by Bert are used as input to perform node and relationship matching with the target knowledge graph G.

[0113] 402. Check whether the matching entity is in G.

[0114] If the matching entity is in the target knowledge graph G, proceed to step 403; if the matching entity is not in the target knowledge graph G, proceed to step 406.

[0115] 403. Whether the matching relationship exists in G.

[0116] If the matching relationship is in the target knowledge graph G, proceed to step 404; if the matching relationship is not in the target knowledge graph G, proceed to step 406.

[0117] 404. Return knowledge check is correct.

[0118] 405. Returns knowledge check error.

[0119] Through secondary inspection of the knowledge graph, the hallucination problem of the output of the large language model is alleviated, the implementation effect of the large language model is improved, and its usability is increased.

[0120] As shown in FIG5 , an embodiment of the present invention provides an output detection device for a large language model, the output detection device for a large language model comprising:

[0121] A first acquisition module 501 is configured to acquire the output text of the large language model for the question text, and acquire a target knowledge graph of the field to which the question text belongs, wherein the target knowledge graph includes knowledge data of the field to which the question text belongs;

[0122] Extraction module 502, configured to perform knowledge extraction on the output text to obtain knowledge data to be detected in the output text;

[0123] The detection module 503 is used to detect the knowledge data to be detected through the target knowledge graph to obtain the knowledge detection result of the output text.

[0124] Optionally, the device further includes:

[0125] The second acquisition module is used to acquire entity words and relationship words in different fields.

[0126] The first knowledge processing module is used to label the entity words and the relationship words in a field through the relationship words to obtain knowledge data, wherein the knowledge data includes at least two entity words and at least one relationship word associated with at least two entity words.

[0127] The second knowledge processing module is used to construct knowledge graphs in different fields based on the knowledge data in different fields.

[0128] Optionally, the first knowledge processing module is further used to divide the entity word-grams into subject word-grams and object word-grams; and to label the subject word-grams and the object word-grams based on the relationship word-grams to obtain knowledge data.

[0129] Optionally, the device further includes:

[0130] A third acquisition module is configured to acquire a natural language processing model to be trained and a training dataset, wherein the training dataset includes sample text and knowledge labels of the sample text, the input of the natural language processing model to be trained is constructed as the sample text, and the output of the natural language processing model to be trained is constructed as predicted knowledge data;

[0131] A first processing module, configured to input the sample text into the natural language processing model to obtain predicted knowledge data of the sample text;

[0132] A second processing module is used to calculate the loss value between the predicted knowledge data of the sample text and the knowledge label of the sample text;

[0133] The third processing module is used to adjust the parameters of the natural language processing model to be trained based on the loss value, and iterate the parameter adjustment process. After the training is completed, a trained natural language processing model is obtained, and the trained natural language processing model is used to extract knowledge from the output text.

[0134] Optionally, the device further includes:

[0135] a fourth acquisition module, configured to acquire a natural language processing model to be trained and a training dataset, wherein the training dataset includes sample text and entity labels and relationship labels of the sample text, wherein the input of the natural language processing model to be trained is constructed as the sample text, and the output of the natural language processing model to be trained is constructed as predicted entities and predicted relationships;

[0136] a fourth processing module, configured to input the sample text into the natural language processing model to obtain predicted entity data and predicted relationship data of the sample text;

[0137] a fifth processing module, configured to calculate a first loss value between the predicted entity data of the sample text and the entity label of the sample text, and a second loss value between the predicted relationship data of the sample text and the relationship label of the sample text;

[0138] The sixth processing module is used to adjust the parameters of the natural language processing model to be trained based on the first loss value and the second loss value, and iterate the parameter adjustment process. After the training is completed, a trained natural language processing model is obtained, and the trained natural language processing model is used to extract knowledge from the output text.

[0139] Optionally, the extraction module 502 is also used to perform knowledge extraction on the output text through the trained natural language processing model to obtain the entities to be detected and the relationships to be detected of the output text; and combine the entities to be detected and the relationships to be detected to obtain the knowledge data of the output text, wherein the knowledge data to be detected includes the entities to be detected and the relationships to be detected.

[0140] Optionally, the detection module 503 is also used to determine whether the entity to be detected and the relationship to be detected exist in the target knowledge graph; if the entity to be detected exists in the target knowledge graph, and the relationship to be detected also exists in the target knowledge graph, then it is determined that the knowledge detection of the output text is correct; if the entity to be detected does not exist in the target knowledge graph, and / or the relationship to be detected does not exist in the target knowledge graph, then it is determined that the knowledge detection of the output text is incorrect.

[0141] The output detection device for a large language model provided in the embodiment of the present invention can implement each process of the output detection method for a large language model in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0142] 6 , which is a schematic structural diagram of an electronic device provided in an embodiment of the present invention. As shown in FIG6 , the electronic device includes: a memory 602, a processor 601, and a computer program for a large language model output detection method stored in the memory 602 and executable on the processor 601, wherein:

[0143] The processor 601 is configured to call the computer program stored in the memory 602 and execute the following steps:

[0144] Obtaining an output text of the large language model for the question text, and obtaining a target knowledge graph of the field to which the question text belongs, wherein the target knowledge graph includes knowledge data of the field to which the question text belongs;

[0145] Performing knowledge extraction on the output text to obtain knowledge data to be detected of the output text;

[0146] The knowledge data to be detected is detected through the target knowledge graph to obtain the knowledge detection result of the output text.

[0147] Optionally, before obtaining the target knowledge graph of the field to which the question text belongs, the method executed by the processor 601 further includes:

[0148] Obtain entity tokens and relationship tokens from different fields;

[0149] For the entity words and the relationship words in a field, different entity words are annotated by the relationship words to obtain knowledge data, wherein the knowledge data includes at least two entity words and at least one relationship word associated with at least two entity words;

[0150] Construct knowledge graphs in different fields based on the knowledge data in different fields.

[0151] Optionally, the processor 601 performs the tagging of different entity word-grams using the relationship word-grams to obtain knowledge data, including:

[0152] Dividing the entity word into subject word and object word;

[0153] The subject word-gram and the object word-gram are labeled based on the relation word-gram to obtain knowledge data.

[0154] Optionally, before performing knowledge extraction on the output text to obtain the knowledge data to be detected of the output text, the method executed by the processor 601 further includes:

[0155] Obtaining a natural language processing model to be trained and a training data set, wherein the training data set includes sample text and knowledge labels of the sample text, wherein the input of the natural language processing model to be trained is constructed as the sample text, and the output of the natural language processing model to be trained is constructed as predicted knowledge data;

[0156] Inputting the sample text into the natural language processing model to obtain predicted knowledge data of the sample text;

[0157] Calculating a loss value between the predicted knowledge data of the sample text and the knowledge label of the sample text;

[0158] The parameters of the natural language processing model to be trained are adjusted based on the loss value, and the parameter adjustment process is iterated. After the training is completed, a trained natural language processing model is obtained, and the trained natural language processing model is used to extract knowledge from the output text.

[0159] Optionally, before performing knowledge extraction on the output text to obtain the knowledge data to be detected of the output text, the method executed by the processor 601 further includes:

[0160] Obtaining a natural language processing model to be trained and a training dataset, wherein the training dataset includes sample text and entity labels and relationship labels of the sample text, wherein the input of the natural language processing model to be trained is constructed as the sample text, and the output of the natural language processing model to be trained is constructed as predicted entities and predicted relationships;

[0161] Inputting the sample text into the natural language processing model to obtain predicted entity data and predicted relationship data of the sample text;

[0162] Calculating a first loss value between the predicted entity data of the sample text and the entity label of the sample text, and a second loss value between the predicted relationship data of the sample text and the relationship label of the sample text;

[0163] Based on the first loss value and the second loss value, the parameters of the natural language processing model to be trained are adjusted, and the parameter adjustment process is iterated. After the training is completed, a trained natural language processing model is obtained, and the trained natural language processing model is used to extract knowledge from the output text.

[0164] Optionally, the processor 601 performs knowledge extraction on the output text to obtain knowledge data to be detected of the output text, including:

[0165] Performing knowledge extraction on the output text using the trained natural language processing model to obtain entities to be detected and relationships to be detected in the output text;

[0166] The entities to be detected and the relationships to be detected are combined to obtain knowledge data of the output text, where the knowledge data to be detected includes the entities to be detected and the relationships to be detected.

[0167] Optionally, the processor 601 performs the detection of the knowledge data to be detected by using the target knowledge graph to obtain the knowledge detection result of the output text, including:

[0168] Determine whether the entity to be detected and the relationship to be detected exist in the target knowledge graph;

[0169] If the entity to be detected exists in the target knowledge graph, and the relationship to be detected also exists in the target knowledge graph, then it is determined that the knowledge detection of the output text is correct;

[0170] If the entity to be detected does not exist in the target knowledge graph, and / or the relationship to be detected does not exist in the target knowledge graph, it is determined that the knowledge detection error of the output text is wrong.

[0171] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the output detection method of the large language model provided in the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0172] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The computer-readable storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0173] The above disclosure is merely a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. An output detection method for a large language model, characterized in that, Including: Obtaining the output text of the large language model for the question text, and obtaining the target knowledge graph of the field to which the question text belongs, the target knowledge graph including the knowledge data of the field to which the question text belongs; Performing knowledge extraction on the output text to obtain the knowledge data to be detected of the output text; Detecting the knowledge data to be detected through the target knowledge graph to obtain the knowledge detection result of the output text.

2. The method according to claim 1, characterized in that, Before obtaining the target knowledge graph of the field to which the question text belongs, the method further includes: Obtaining entity tokens and relationship tokens in different fields; For the entity tokens and the relationship tokens in one field, performing annotation between different entity tokens through the relationship tokens to obtain knowledge data, the knowledge data including at least two entity tokens and at least one relationship token associating the at least two entity tokens; Constructing knowledge graphs of different fields according to the knowledge data of different fields.

3. The method according to claim 2, characterized in that, The performing annotation between different entity tokens through the relationship tokens to obtain knowledge data includes: Dividing the entity tokens into subject tokens and object tokens; Based on the relationship tokens, performing annotation between the subject tokens and the object tokens to obtain knowledge data.

4. The method according to claim 1, wherein Before performing knowledge extraction on the output text to obtain the knowledge data to be detected of the output text, the method further includes: Obtaining a natural language processing model to be trained and a training data set, the training data set including sample texts and knowledge labels of the sample texts, the input of the natural language processing model to be trained being constructed as the sample texts, and the output of the natural language processing model to be trained being constructed as predicted knowledge data; Inputting the sample texts into the natural language processing model to obtain the predicted knowledge data of the sample texts; Calculating the loss value between the predicted knowledge data of the sample texts and the knowledge labels of the sample texts; Based on the loss value, adjusting the parameters of the natural language processing model to be trained and iterating the parameter adjustment process. After training is completed, obtaining a trained natural language processing model, the trained natural language processing model being used to perform knowledge extraction on the output text.

5. The method according to claim 1, characterized in that, Before performing knowledge extraction on the output text to obtain the knowledge data to be detected of the output text, the method further includes: Obtaining a natural language processing model to be trained and a training data set, the training data set including sample texts and entity labels and relationship labels of the sample texts, the input of the natural language processing model to be trained being constructed as the sample texts, and the output of the natural language processing model to be trained being constructed as predicted entities and predicted relationships; Inputting the sample texts into the natural language processing model to obtain the predicted entity data and predicted relationship data of the sample texts; Calculating a first loss value between the predicted entity data of the sample texts and the entity labels of the sample texts, and a second loss value between the predicted relationship data of the sample texts and the relationship labels of the sample texts; Based on the first loss value and the second loss value, adjust the parameters of the natural language processing model to be trained, and iterate the parameter adjustment process. After the training is completed, obtain a trained natural language processing model, which is used to extract knowledge from the output text.

6. The method according to claim 5, wherein The extracting knowledge from the output text to obtain the to-be-detected knowledge data of the output text includes: Using the trained natural language processing model to extract knowledge from the output text to obtain the to-be-detected entities and to-be-detected relationships of the output text; Combining the to-be-detected entities and to-be-detected relationships to obtain the knowledge data of the output text, where the to-be-detected knowledge data includes to-be-detected entities and to-be-detected relationships.

7. The method according to claim 6, wherein The detecting the to-be-detected knowledge data through the target knowledge graph to obtain the knowledge detection result of the output text includes: Determine whether the to-be-detected entities and the to-be-detected relationships exist in the target knowledge graph; If the to-be-detected entities exist in the target knowledge graph and the to-be-detected relationships also exist in the target knowledge graph, determine that the knowledge detection of the output text is correct; If the to-be-detected entities do not exist in the target knowledge graph, and / or the to-be-detected relationships do not exist in the target knowledge graph, determine that the knowledge detection of the output text is incorrect.

8. An output detection device for a large language model, characterized in that, The output detection device of the large language model includes: A first acquisition module, configured to acquire the output text of the large language model for the question text, and acquire the target knowledge graph of the field to which the question text belongs, where the target knowledge graph includes the knowledge data of the field to which the question text belongs; An extraction module, configured to extract knowledge from the output text to obtain the to-be-detected knowledge data of the output text; data; A detection module, configured to detect the to-be-detected knowledge data through the target knowledge graph to obtain the knowledge detection result of the output text.

9. An electronic device, characterized in that, including: A memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, the steps in the output detection method of the large language model according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the output detection method of the large language model according to any one of claims 1 to 7 are implemented.

Images