Model verification method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a model verification method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: inputting a first text into a first text generation model for text generation, and obtaining a first response text of the first text; generating a second text according to the first response text or the first text; inputting the second text into the first text generation model for text generation to obtain a second response text of the second text; and determining a verification result of the first text generation model according to the first response text and the second response text. Therefore, whether the first text generation model is reliable or not can be efficiently verified.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a model verification method, device, electronic device, storage medium and program product. Background Art

[0002] With the development of electronic technology, the application of NLP (Natural Language Processing) technology is becoming more and more widespread. Language model is one of the commonly used tools for processing NLP tasks.

[0003] Some language models have the ability to generate text and can respond to text input into the model. The essence of this response can be understood as the accumulation of some knowledge reserves during the model training phase, and the use of the accumulated knowledge reserves to complete the text input into the model during the response process, rather than performing data query operations based on the text input into the model and responding to the text input into the model based on the query results. Summary of the invention

[0004] The embodiments of the present application provide a model verification method, device, electronic device, storage medium and program product, which can efficiently verify whether the first text generation model is reliable.

[0005] In a first aspect, an embodiment of the present application provides a model verification method, comprising:

[0006] Inputting the first text into a first text generation model to generate text, thereby obtaining a first response text to the first text;

[0007] Generate a second text according to the first response text or the first text;

[0008] Inputting the second text into the first text generation model to generate text, thereby obtaining a second response text for the second text;

[0009] A verification result of the first text generation model is determined based on the first response text and the second response text.

[0010] In a second aspect, an embodiment of the present application provides a model verification device, including:

[0011] A generating unit, configured to input a first text into a first text generating model for text generating processing, and obtain a first response text of the first text;

[0012] The generating unit is further configured to generate a second text according to the first response text or the first text;

[0013] The generating unit is further configured to input the second text into the first text generating model for text generating processing to obtain a second response text of the second text;

[0014] A determination unit is used to determine a verification result of the first text generation model according to the first response text and the second response text.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein when the computer-executable instructions are executed, the processor executes the model verification method as described in the first aspect.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing computer-executable instructions, which, when executed by a processor, implement the model verification method as described in the first aspect.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the model verification method as described in the first aspect.

[0018] It can be seen that in the embodiment of the present application, first, the first text is input into the first text generation model for text generation to obtain the first answer text of the first text; then, according to the first answer text or the first text, the second text is generated; then, the second text is input into the first text generation model for text generation to obtain the second answer text of the second text; finally, according to the first answer text and the second answer text, the verification result of the first text generation model is determined. In this way, considering that the essence of the text generation of the first text generation model is to complete the input text, rather than data query, therefore, the text content of the first answer text obtained by the first text generation model according to the input first text for text generation may be true and reliable, or it may be fabricated by the first text generation model, by inputting the second text generated according to the first answer text or the first text into the first text generation model for text generation to obtain the second answer text, and then, the first answer text and the second answer text can mutually verify the authenticity of each other's text content. For any text generation model, the more frequently the model outputs wrong content, the lower its accuracy, the more unreliable the model, the rarer the model outputs wrong content, the higher its accuracy, and the more reliable the model. Therefore, by mutually verifying the authenticity of the other party's text content through the first answer text and the second answer text, it is possible to verify whether the first text generation model is reliable. In addition, in the entire verification process, the second text is not manually written, but generated according to the first answer text or the first text, which reduces manual participation and improves the efficiency of dialogue, thereby indirectly improving the model verification efficiency. Therefore, the model verification method provided in the embodiment of the present application can efficiently verify whether the first text generation model is reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

[0020] Figure 1 A schematic diagram of an implementation environment of a model verification method provided in an embodiment of the present application;

[0021] Figure 2 A processing flow chart of a model verification method provided in an embodiment of the present application;

[0022] Figure 3 A processing flow chart of another model verification method provided in an embodiment of the present application;

[0023] Figure 4A schematic diagram of a model verification device provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] This specification provides a model verification method embodiment:

[0027] In consulting scenarios, using language models to replace human agents in conversations with customers can save a lot of human resources, reduce customer waiting time, and solve problems quickly and efficiently. However, language models may fabricate false data due to insufficient knowledge reserves, which will obviously mislead users who ask questions and cause customer dissatisfaction. Therefore, before a language model is put into use, it is often necessary to verify whether the model is reliable. If it is determined that the model often fabricates false data, the model needs to be further trained.

[0028] In order to verify whether a language model is reliable, testers are often required to communicate with the language model, which is inefficient. Therefore, the issue of how to efficiently verify whether a language model is reliable has received increasing attention.

[0029] Therefore, in order to solve the above problems, an embodiment of the present application provides a model verification method.

[0030] The model verification method provided in one or more embodiments of this specification may be applicable to an implementation environment of the model verification method, which implementation environment at least includes a server 101 for verifying the first text generation model.

[0031] The server 101 may be a single server, or a server cluster consisting of a plurality of servers, or one or more cloud servers in a cloud computing platform, and is used to verify the first text generation model.

[0032] In this implementation environment, during the process of verifying the first text generation model, the server 101 first inputs the first text into the first text generation model for text generation to obtain the first answer text of the first text; then, according to the first answer text or the first text, the second text is generated; then, the second text is input into the first text generation model for text generation to obtain the second answer text of the second text; finally, according to the first answer text and the second answer text, the verification result of the first text generation model is determined. In this way, considering that the essence of the text generation of the first text generation model is to complete the input text, rather than data query, the text content of the first answer text obtained by the first text generation model for text generation according to the input first text may be true and reliable, or it may be fabricated by the first text generation model. By inputting the second text generated according to the first answer text or the first text into the first text generation model for text generation to obtain the second answer text, the first answer text and the second answer text can mutually verify the authenticity of each other's text content. For any text generation model, the more frequently the model outputs wrong content, the lower its accuracy and the more unreliable the model is; the rarer the model outputs wrong content, the higher its accuracy and the more reliable the model is. Therefore, by mutually verifying the authenticity of the other party's text content through the first answer text and the second answer text, it is possible to verify whether the first text generation model is reliable. In addition, in the entire verification process, the second text is not manually written, but generated according to the first answer text or the first text, which reduces manual participation and improves the efficiency of dialogue, thereby indirectly improving the model verification efficiency. Therefore, the model verification method provided in the embodiment of the present application can efficiently verify whether the first text generation model is reliable.

[0033] Figure 2 A processing flow chart of a model verification method provided in an embodiment of the present application. Figure 2 The model verification method provided in this embodiment specifically includes steps S202 to S210:

[0034] Step S202: input the first text into a first text generation model to generate text, thereby obtaining a first response text of the first text.

[0035] The first text may be any text. The number of the first texts may be one or more.

[0036] The ordinal numbers such as "first" and "second" appearing in this specification are used to distinguish different technical features and have no actual meaning and will not be elaborated below.

[0037] During specific implementation, the first text may be a question text, a request text, an instruction text, and so on.

[0038] For example, the first text is "When is the deadline for activity A?" The first text is input into the first text generation model for text generation processing to obtain the first response text of the first text "Hello, the deadline for activity A is X year Y month Z day."

[0039] For another example, the first text is "I want to inquire about the progress of business A", the first text is input into the first text generation model for text generation processing, and the first response text of the first text is obtained as "OK, please provide your business order number".

[0040] Text generation refers to the use of natural language processing technology to automatically generate text content that meets grammatical and semantic requirements by learning and understanding a large amount of text data and mastering language rules. Text generation is an important application field of natural language processing technology.

[0041] The first text generation model may be any language model with text generation capability, for example, a GPT-3 (Generative Pre-trained Transformer-3) model, a bloom model, and the like.

[0042] In the field of artificial intelligence, for any model, after model training and before the model is officially put into use, it may be necessary to verify the model to determine whether the model is suitable for application.

[0043] The language model has accumulated some knowledge reserves during the training phase. In the process of generating texts through the language model to answer users' questions, it is actually using the language model's own knowledge reserves to complete the text of the questions. The number of parameters of some language models has gradually expanded to tens of billions to form a large model. In the process of text completion of this large model, AI (Artificial Intelligence) illusions may occur.

[0044] The large model, also known as the Foundation Model, extracts knowledge from billions of corpora or images, learns and produces a large model with billions of parameters, and uses a large amount of unlabeled and very cheap data for pre-training.

[0045] AI hallucinations occur when the answers provided by a language model appear reasonable but are completely wrong. These inaccurate answers are not intentional lies by the language model, but rather because the training data used to train the model does not contain the necessary information to correctly answer the question. In some cases, the language model may explicitly indicate that it does not know the answer, while in other cases, the language model may generate completely fictitious answers, which is called AI hallucinations.

[0046] If a language model is put into use, and it frequently produces artificial hallucinations during text generation and processing, it is likely to mislead users who are conversing with the language model and reduce the user experience. Therefore, before a language model is put into use, it is often necessary to verify whether the frequency with which the language model produces AI hallucinations meets the preset requirements. For example, the preset requirements stipulate that the language model is only allowed to be put into use if the frequency with which the language model produces AI hallucinations is less than a preset frequency threshold.

[0047] The first text may be text input by a user, or may be text generated by another language model other than the first text generation model.

[0048] In the case where the first text is a text generated by another language model, the probability of the other language model generating AI hallucination is less than a preset probability threshold, and the other language model can be regarded as a model with high accuracy, that is, a relatively reliable model.

[0049] The other language model may be a language model that has text generation capability and has been put into use.

[0050] For example, two robots are divided into two roles, robot A is assigned to the customer role, and robot B is assigned to the agent role. Robot B includes a first text generation model, and robot A includes another language model other than the first text generation model.

[0051] Robot A sends a text text1, which is input into the model of robot B. Robot B outputs text2 as a reply to text1. Text2 is input into the model of robot A. Robot A outputs text3 as a reply to text2. This cycle repeats to form a conversation.

[0052] The number of first text generation models may be one or more. In the case of multiple first text generation models, considering that the speed at which a language model generates text is much higher than the speed of human conversation, communicating with the first text generation model through another language model is more conducive to improving model verification efficiency than manually communicating with the first text generation model.

[0053] In addition, before performing text generation processing according to the first text by the first text generation model, the text input by the user or the text output by another language model may be combined with prompt information.

[0054] Prompt Learning is a paradigm that uses designed prompts to guide machine learning. It significantly improves performance by adding prompt words to the input of pre-trained models. It is suitable for low-resource scenarios and can achieve good task results even without a large number of samples.

[0055] At present, the pre-trained language model has entered the era of prompt mode. Taking the text generation model as an example, the previous models often input a text, and the text generation model outputs a text as an answer; in the prompt mode, a prompt message is added after the input text. This prompt message can be one or more, which can play a role in limiting and guiding the reply.

[0056] In a specific implementation, inputting the first text into the first text generation model for text generation to obtain a first response text for the first text may include: receiving user input, performing splicing processing based on the user input and the first prompt to obtain the first text, and inputting the first text into the first text generation model for text generation to obtain a first response text for the first text.

[0057] Inputting the first text into the first text generation model for text generation to obtain a first response text for the first text may also include: receiving the dialogue text output by the language model, splicing the dialogue text with the first prompt to obtain the first text, and inputting the first text into the first text generation model for text generation to obtain a first response text for the first text.

[0058] The first prompt may be a custom-configured prompt text, which may assist the first text generation model to better perform text generation processing.

[0059] In a specific implementation, before the first text is input into the first text generation model for text generation to obtain a first response text of the first text, the model verification method also includes: receiving a fourth text; the fourth text is obtained by text generation through the third text generation model; determining a prompt text corresponding to the fourth text; and splicing the fourth text with the prompt text to obtain the first text.

[0060] The third text generation model may be another language model other than the first text generation model. The third text generation model has text generation capability, and the frequency of the third text generation model generating AI hallucinations is less than a preset frequency threshold.

[0061] The fourth text may be randomly generated by the third text generation model, or may be obtained by inputting a preset initial text into the third text generation model for text generation processing, or may be obtained by inputting a previous text output by the first text generation model into the third text generation model for text generation processing, wherein the previous text refers to the text generated by the first text generation model before the first answer text. The preset initial text may be custom configured.

[0062] In specific implementation, the prompt information corresponding to the fourth text is determined by performing intent recognition on the fourth text and adding the intent recognition result to the prompt information, or by performing intent recognition on the fourth text, querying the target prompt text from the correspondence between the pre-configured intent and the prompt text according to the intent recognition result, and adding the target prompt text to the prompt information.

[0063] For example, the pre-configured correspondence between intent and prompt text includes:

[0064] meaning Figure 1 <——>Prompt text 1;

[0065] meaning Figure 2 <——>Prompt text 2;

[0066] meaning Figure 3 <——>Prompt text 3;

[0067] meaning Figure 4 <——>Prompt text 4.

[0068] The fourth text is "I can repay the loan", and the intent recognition result is Figure 2 . According to your opinion Figure 2 A query process is performed in the correspondence between the above intention and the prompt text to obtain the target prompt text: prompt text 2. The prompt text 2 is added to the prompt information corresponding to the fourth text.

[0069] By adding the intent recognition result or the target prompt text to the prompt information, a restriction and guidance effect can be exerted on the response of the first text generation model to the first text, which is conducive to reducing the situation where the first text generation model answers irrelevant questions.

[0070] In a specific implementation, the prompt information corresponding to the fourth text may be determined by detecting keywords in the fourth text, locating the target object according to the keywords, querying the portrait information of the target object, and adding the portrait information to the prompt information.

[0071] The target object may be one or more of a preset object set. The preset object set may include multiple objects. An object may be a person, an item, an account, etc. The portrait information of the target object is used to describe the characteristic information of the target object in one or more dimensions.

[0072] For example, the target object is user 1, and the portrait information of user 1 includes user 1's age, gender, occupation, education level, preference tags, etc.

[0073] In addition, considering that the portrait information of the target object may involve user privacy, the portrait information can be obtained with the authorization of the target object.

[0074] By adding the portrait information to the prompt information, it is helpful for the first text generation model to generate personalized text for the target object when performing text generation processing under the prompt of the portrait information.

[0075] In a specific implementation, determining the prompt information corresponding to the fourth text may also be detecting keywords in the fourth text, determining auxiliary information in the historical conversation according to the keywords, adding the auxiliary information to the prompt information, and so on.

[0076] For example:

[0077] Fourth text: I can repay. Prompt message 1: 1,000 yuan, before 6 o'clock.

[0078] First response text: OK, please deposit 1,000 yuan on the repayment card before 6 o'clock, and we will initiate the deduction then.

[0079] In the above example, the keyword in the fourth text includes "repayment", and the auxiliary information "1,000 yuan" and "before 6 o'clock" are determined in the historical dialogue based on "repayment", and the auxiliary information is added to the prompt information 1.

[0080] Considering that in spoken conversations, real users or virtual users simulated by robots often do not speak long sentences with complete elements, short sentences are more commonly used, and some feature information (for example, amount, date) has been mentioned once in historical conversations and will not be mentioned repeatedly in every sentence, when the prompt information includes auxiliary information, the prompt information is used to supplement the fourth text, which is conducive to the first text generation model to better respond to the first text.

[0081] The prompt information corresponding to the fourth text may include one or more of the intention recognition result, the target prompt text, the portrait information, and the auxiliary information.

[0082] According to the splicing processing performed on the fourth text and the prompt text, the fourth text may be spliced ​​before the prompt text or after the prompt text.

[0083] Step S204: Generate a second text according to the first response text or the first text.

[0084] Generate a second text based on the first response text; or generate a second text based on the first text.

[0085] Exemplarily, the second text may be a question text / request text / instruction text, and the like.

[0086] In a specific implementation, the second text includes a first sub-text; based on the first response text, the second text is generated, including: extracting information from the first response text to obtain an information extraction result; when the information extraction result includes a third text indicating that the first response text has an abnormal risk, inputting the third text into a second text generation model for text generation to obtain the first sub-text; the first sub-text is used to question the information source of the third text.

[0087] Taking into account that the second text can be generated based on the first response text or the first text, in order to facilitate the distinction between the second text generated based on the first response text and the second text generated based on the first text, in the present implementation, the second text includes a first sub-text. In the process of generating the second text based on the first response text, the following steps can be performed: extract information from the first response text to obtain an information extraction result; when the information extraction result includes a third text indicating that the first response text has an abnormal risk, input the third text into the second text generation model for text generation to obtain a first sub-text; the first sub-text is used to question the information source of the third text.

[0088] Information extraction is performed on the first response text to obtain an information extraction result.

[0089] IE (Information Extraction) is to structure the information contained in the text into a table-like organizational form. The input information of the extraction system is the original text, and the output is information points in a fixed format. Information points are extracted from various documents and then integrated in a unified form. This is the main task of information extraction. The advantage of integrating information in a unified form is that it is convenient for inspection and comparison. Information extraction technology does not attempt to fully understand the entire document, but only analyzes the part of the document that contains relevant information. As for which information is relevant, it will be determined by the scope of the field determined when the system is designed.

[0090] Information extraction technology is a technology that converts unstructured text into structured information. This technology can automatically extract entities, relationships and other important information from various documents and convert them into a format that is convenient for computers to further process and analyze.

[0091] Information extraction includes: named entity recognition, relationship extraction and event extraction. Named entity recognition refers to the recognition of some meaningful words in the text, such as names of people, places, and institutions. Relation extraction refers to the recognition of entities with relationships in the text and the relationships between them, such as (address 1, address 2). Event extraction refers to the recognition of events and related arguments in the text. Events generally have event trigger words, and event arguments are some entities that describe the event.

[0092] Information extraction for the first response text may be performed by performing keyword extraction processing from the first response text. For example, it is predetermined that digital type data is used as a keyword, and when digital type data is detected, the digital type data is extracted as an information extraction result.

[0093] The first response text may be subjected to information extraction, or the first response text may be input into an information extraction model for information extraction processing to obtain an information extraction result.

[0094] There are two possible situations when performing information extraction operations:

[0095] In the first case, the first response text is input into the information extraction model for information extraction to obtain a third text; the third text may be an amount, a historical event, and so on.

[0096] In the second case, the first answer text is input into the information extraction model for information extraction, and a null value is obtained, that is, no text data is extracted. In practical applications, the second case is very likely to occur. For example, the first answer text does not include information such as amount, historical events, etc., and there is no risk of AI hallucination. In this case, the information extraction result obtained by extracting information from the first answer text is a null value.

[0097] In the first case mentioned above, the information extraction result includes a third text in which the first response text has an abnormal risk. The third text is input into the second text generation model for text generation to obtain a first sub-text; the first sub-text is used to question the information source of the third text.

[0098] In the second case mentioned above, the information extraction result does not include the third text, and it can be determined that the first response text does not have the risk of producing AI hallucinations. Therefore, the first response text cannot be used for model verification, and there is no need to execute subsequent steps S206-S208.

[0099] Regarding "the third text in the first response text has abnormal risks", the existence of abnormal risks means that, in the process of inputting the first text into the first text generation model for text generation to obtain the first response text, there may be a risk that the first text generation model produces AI hallucinations and fabricates false data, and the authenticity of the text content of the first response text is questionable. Specifically, the text content of the third text in the first response text may be false data fabricated by the first text generation model, or it may be true and reliable, which requires further verification.

[0100] For example, if it is predetermined that the text content whose authenticity may be questionable involves amounts and historical events, then when training the information extraction model, the amounts and historical events in the training samples can be labeled first, so that the information extraction model will learn how to extract amounts and historical events from the text. Then, in the process of inputting the first response text into the information extraction model for information extraction, if the first response text includes amounts and / or historical events, the information extraction model will extract this part as the third text. If the first response text does not include any amounts or historical events, the information extraction model will output a null value, that is, no third text is extracted.

[0101] The information extraction model can be trained in the following way:

[0102] (a1) Prepare the labeled data and label the data format.

[0103] The text samples are labeled for the information to be extracted. The information to be extracted may be a phrase that is likely to cause AI hallucinations, determined based on the statistical results of historical data. Here, likely to cause AI hallucinations may refer to the ratio of the number of times the phrase produces AI hallucinations to the number of times the phrase appears being greater than or equal to a preset ratio threshold. The information to be extracted may be an entity, an event, a relationship, and so on.

[0104] Taking the financial field as an example, the information to be extracted may include amount, historical events, and so on.

[0105] For example, text sample: You owe four thousand yuan; label: [3, 5, amount].

[0106] Among them, "3" and "5" in the label are used to indicate the position information of the information to be extracted "four thousand yuan" in the text sample. "Amount" in the label is used to indicate the target data type of the information to be extracted "four thousand yuan". The target data type is one of the preset data type sets.

[0107] Specifically, the position information of "you" in the text sample can be represented by "0", the position information of "owe" in the text sample can be represented by "1", the position information of "money" in the text sample can be represented by "2", the position information of "four" in the text sample can be represented by "3", the position information of "thousand" in the text sample can be represented by "4", and the position information of "yuan" in the text sample can be represented by "5". The "3" in the label is used to represent the position information of the first character "four" in "four thousand yuan" in the text sample, and the "5" in the label is used to represent the position information of the last character "yuan" in "four thousand yuan" in the text sample.

[0108] (a2) Input the labeled data into the information extraction model to be trained for model training.

[0109] Exemplarily, the information extraction model to be trained can be a Flat model.

[0110] The Flat model is a Chinese NER (Named Entity Recognition) model based on Flat-Lattice Transformer (flattened lattice framework transformer). The Flat-Lattice Transformer is an extended Transformer architecture. Transformer is a neural network architecture based on the attention mechanism.

[0111] The trained Flat model can be used for information extraction processing. Specifically, it can perform one or more of named entity recognition, relation extraction, and event extraction. The trained Flat model is the aforementioned information extraction model.

[0112] Input the first response text into the information extraction model for information extraction to obtain an information extraction result. For example:

[0113] The first response text is: What is the specific use of the three hundred and fifty yuan you mentioned yesterday?

[0114] Input the first response text into the information extraction model for information extraction processing. The obtained information extraction result is: ["mentioned yesterday", "three hundred and fifty yuan"]. Among them, "mentioned yesterday" is a sub-text of the first response text, and the data type of this sub-text is "historical event", and "three hundred and fifty yuan" is another sub-text of the first response text, and the data type of this sub-text is "amount of money".

[0115] If the text classification result of the first response text indicates that the first response text is a second type of text, it is determined that the information extraction result of the first response text includes at least one sub-text, and then steps S206 to S210 may be continued for the first response text.

[0116] For example, the first response text 1 is: What is the specific purpose of the 350 yuan you mentioned yesterday?

[0117] The first response text 1 is subjected to text classification processing, and the obtained text classification result is: [second type of text].

[0118] The first response text 2 is: We are sorry that we cannot solve your problem.

[0119] The first response text 2 is subjected to text classification processing, and the obtained text classification result is: [first type of text].

[0120] When the information extraction result includes a third text indicating that the first response text has an abnormal risk, the third text is input into the second text generation model for text generation to obtain a first subtext.

[0121] If the information extraction result of the first response text includes a third text, the third text is input into the second text generation model for text generation to obtain a first sub-text corresponding to the third text.

[0122] If the information extraction result of the first response text includes multiple third texts, for each third text in the multiple third texts, the third text is input into the second text generation model for text generation to obtain a first sub-text corresponding to the third text.

[0123] For example, the information extraction result of the first answer text includes: third text 1, third text 2 and third text 3. The third text 1 is input into the second text generation model for text generation to obtain the first subtext 1; the third text 2 is input into the second text generation model for text generation to obtain the first subtext 2; the third text 3 is input into the second text generation model for text generation to obtain the first subtext 1.

[0124] If the information extraction result of the first response text does not include the third text, there is no need to execute steps S206 to S208.

[0125] The second text generation model may be a language model with text generation capability, for example, a T5 model.

[0126] The T5 (Text-to-Text Transfer Transformer) model is a Transformer-based NLP model.

[0127] The first subtext is used to question the information source of the third text. Information source is the source of information.

[0128] Each third text may describe a feature data, and the feature data may be pre-stored in a sub-region in a preset storage region, and the information source is the sub-region storing the feature data.

[0129] For example, the third text 1 "1000 yuan" describes the selling price of commodity A on trading platform B. The specific value of the selling price is stored in column C of data table B in database A. The first sub-text is "How is 1000 yuan determined". The first sub-text is actually asking which database, which data table and which column the selling price of commodity A described by "1000 yuan" on trading platform B is stored in.

[0130] It should be emphasized that although the first sub-text is used to inquire about the information source of the third text, when the first text generation model generates the first response text based on the first text, it does not perform data query processing based on the information source to obtain the feature data described by the third text, but uses the knowledge reserves accumulated in the training phase to complete the text. Therefore, the feature data described in the third text may be the same as or different from the feature data stored in the information source of the third text.

[0131] The second text generation model can be trained as follows:

[0132] (b1) Prepare training data.

[0133] Text sample 1: 2:30 p.m., Label: How did you determine it was 2:30 p.m.?

[0134] Text sample 2: Ten minutes, label: What is the basis for ten minutes?

[0135] (b2) Inputting the training data into the text generation model to be trained to perform model training, thereby obtaining a second text generation model.

[0136] In a specific implementation, the model verification method also includes: classifying the first response text to obtain a classification result of the first response text; extracting information from the first response text to obtain an information extraction result; if the classification result indicates that the first response text does not include text with abnormal risks, and the information extraction result includes a third text with abnormal risks, determining that an information extraction anomaly exists; if the classification result indicates that the first response text includes text with abnormal risks, and the information extraction result is empty, determining that an information extraction anomaly exists; marking the first response text with information extraction anomaly, and the first text corresponding to the first response text.

[0137] The first response text is classified to obtain a classification result of the first response text, and the classification result of the first response text can be obtained by classifying the first response text through a preset text classification model.

[0138] Information extraction is performed on the first response text to obtain an information extraction result, and the information extraction result can be obtained by performing information extraction on the first response text through a preset information extraction model.

[0139] The classification result of the first response text is used to assist in verifying whether the information extraction is abnormal.

[0140] In addition, the same model can also be used to process classification tasks and information extraction tasks, that is, the input data of the model is the first response text, and the output data includes the classification results and information extraction results of the first response text. The model can be used for both text classification and information extraction.

[0141] The classification result of the text classification may indicate whether the first response text includes text with an abnormal risk.

[0142] In actual applications, there may be various situations as shown below:

[0143] (1) The classification result of the first response text indicates that the first response text does not include text with abnormal risks, and the information extraction result is empty.

[0144] In this case, it can be determined that there is no abnormality in information extraction.

[0145] (2) The classification result of the first response text indicates that the first response text does not include text with abnormal risk, and the information extraction result includes the third text with abnormal risk.

[0146] In this case, it can be determined that there is an information extraction anomaly.

[0147] (3) The classification result of the first response text indicates that the first response text includes text with abnormal risk, and the information extraction result includes the third text with abnormal risk.

[0148] In this case, it can be determined that there is no abnormality in information extraction.

[0149] (4) The classification result of the first response text indicates that the first response text includes text with abnormal risks, and the information extraction result is empty.

[0150] In this case, it can be determined that there is an information extraction anomaly.

[0151] In addition, the reason why whether there is an information extraction anomaly is determined by the classification results and information extraction results of the first answer text instead of determining whether there is a classification anomaly is because in the process of generating the second text based on the first answer text, the importance of information extraction is far greater than text classification.

[0152] If it is determined that there is an information extraction anomaly, the first response text with the information extraction anomaly and the first text corresponding to the first response text are marked, which facilitates the subsequent analysis of the cause of the information extraction error.

[0153] In a specific implementation, the second text includes a second subtext; generating the second text according to the first text includes: determining a first character to be blocked in the first text; replacing the first character in the first text by a preset blocking character to obtain the second subtext.

[0154] Taking into account that the second text can be generated based on the first response text or the first text, in order to facilitate the distinction between the second text generated based on the first response text and the second text generated based on the first text, in the present implementation, the second text includes a second sub-text. In the process of generating the second text based on the first text, the following steps can be performed: determining the first character to be blocked in the first text; replacing the first character in the first text by presetting the blocking character to obtain the second sub-text.

[0155] When the first text is obtained by concatenating a fourth text and a prompt text corresponding to the fourth text, and the prompt text includes at least one phrase, determining the first character to be obscured in the first text may be by determining a target phrase in the at least one aforementioned phrase, and determining the first character to be obscured in the first text based on the target phrase.

[0156] For example: the first text is obtained by concatenating the fourth text and prompt information 1.

[0157] Fourth text: I can repay. Prompt message 1: 1,000 yuan, before 6 o'clock.

[0158] First response text: OK, please deposit 1,000 yuan on the repayment card before 6 o'clock, and we will initiate the deduction then.

[0159] The target phrase is determined to be "before 6 o'clock" among the two phrases included in the prompt information 1. According to the target phrase "before 6 o'clock", the first character to be blocked in the first text is determined to include "6", "dot", and "before".

[0160] When the information extraction result of the first response text includes a third text, and at least one third text describes an event, determining the first character to be obscured in the first text can be performed by randomly selecting an event from each event as a target event, and determining the first character to be obscured in the first text based on the target event.

[0161] By presetting the masking character, the first character is replaced in the first text to obtain the second subtext.

[0162] For example, the first text is ABCDEFG, the first character to be masked includes BC, and the preset masking character is [MASK]

[0163] Through [MASK], BC is replaced in ABCDEFG to obtain A[MASK][MASK]DEFG.

[0164] For another example, the first text is obtained by concatenating the fourth text and prompt information 1.

[0165] Fourth text: I can repay. Prompt message 1: 1,000 yuan, before 6 o'clock.

[0166] First response text: OK, please deposit 1,000 yuan on the repayment card before 6 o'clock, and we will initiate the deduction then.

[0167] The target phrase is determined to be "before 6 o'clock" among the two phrases included in the prompt information 1. According to the target phrase "before 6 o'clock", the first character to be blocked in the first text is determined to include "6", "dot", and "before".

[0168] By presetting the mask character [MASK], the first character in the first text is replaced to obtain the second subtext 1:

[0169] I can repay. 1,000 yuan, [MASK][MASK][MASK].

[0170] The first text and the second sub-text substantially describe the same question / request / instruction. The first text is a complete unobstructed text, and the second sub-text is a text obtained by obstructing partial content on the basis of the first text.

[0171] Step S206: input the second text into the first text generation model to generate text, and obtain a second response text for the second text.

[0172] The second text is input into the first text generation model for text generation to obtain a second response text for the second text. The corresponding description part of step S202 can be referred to.

[0173] The second text may include the first subtext or the second subtext.

[0174] When the second text includes the first sub-text, the second text is input into the first text generation model for text generation to obtain a second response text for the second text. The first sub-text and the second prompt text can be spliced ​​to obtain a first spliced ​​text, and the first spliced ​​text is input into the first text generation model for text generation to obtain a second response text for the second text.

[0175] The first subtext is used to inquire about the information source of the third text, and the details can be referred to the corresponding description part above. The second prompt text can be pre-defined and configured.

[0176] For example, the second prompt text is: Please specify the webpage address and the number of paragraphs for the following question. The first subtext is: How did you determine 350 yuan. The first spliced ​​text is: Please specify the webpage address and the number of paragraphs for the following question: How did you determine 350 yuan?

[0177] The first concatenated text is input into the first text generation model for text generation to obtain a second response text of the second text: the fifth paragraph of the first web page records that the price of XXX is 350 yuan, and the web address of the first web page is L1.

[0178] In addition, in the case where the second text includes the first sub-text, the above-mentioned step S206 can also be replaced by "searching for a standard text with semantics similar to the first sub-text in a preset text set based on the first sub-text, inputting the standard text into the first text generation model for text generation, and obtaining a second response text for the second text", and forming a new implementation method together with the other processing steps in this embodiment.

[0179] The preset text set may include multiple standard texts. In a specific implementation, the standard text with semantics similar to the first sub-text is searched in the preset text set according to the first sub-text. The semantic similarity between each standard text in the preset text set and the first sub-text can be calculated respectively. When the semantic similarity is greater than a preset similarity threshold, the standard text corresponding to the semantic similarity is determined as the standard text with semantics similar to the first sub-text.

[0180] The second response text may include index information characterizing the information source.

[0181] For example, the feature data 1 described in the third text 1 is stored in [column name 1] of the data table [table name 1]. The second response text is: the third text 1 is recorded in [column name 1] of the data table [table name 1]. The second response text includes table name 1 and column name 1.

[0182] The second response text may also include address information identifying the source of the information.

[0183] For example, the feature data 2 described by the third text 2 is stored in the Nth segment of the web page 1, N is an integer greater than or equal to 1, and the web page address of the web page 1 is L1. The second response text is: the Nth segment of the web page 1 records the subtext 2, and the address of the web page 1 is L1. The second response text includes L1 and N.

[0184] The second response text may also include description information characterizing the information source.

[0185] For example, the feature data 3 described by the third text 3 is stored in the Kth line of the historical conversation on the first date in the historical conversation set, where K is an integer greater than or equal to 1. The second response text is: User A mentioned the third text 3 in the Kth line of the historical conversation on the first date. The second response text includes: the historical conversation on the first date and the Kth line.

[0186] In the case where the second text includes a second sub-text, the second text is input into the first text generation model for text generation to obtain a second response text for the second text. The second sub-text and the aforementioned prompt text may be concatenated to obtain a second concatenated text, and the second concatenated text is input into the first text generation model for text generation to obtain a second response text for the second text.

[0187] The definitions of the second subtext and the prompt text may refer to the corresponding description parts mentioned above.

[0188] It should be noted that, since the first text and the second sub-text essentially describe the same question / request / instruction, the prompt text spliced ​​with the second sub-text during text generation is the prompt text included in the first text.

[0189] Step S208: Determine the verification result of the first text generation model according to the first response text and the second response text.

[0190] When the second text is generated based on the first response text, that is, the second text includes the first sub-text, the second response text can describe the information source of the third text, and the reference text of the third text can be queried based on the information source. If the third text is the same as the reference text, it means that the text content output by the first text generation model is accurate, that is, the first text generation model did not produce AI hallucination when processing the first text; if the third text is different from the reference text, it means that the text content output by the first text generation model is inaccurate, that is, the first text generation model produced an AI hallucination when processing the first text.

[0191] The third text may describe a characteristic data, for example, the amount of commodity A, the location of event B, etc. The reference text of the third text describes the same characteristic data as the third text. The storage location of the reference text of the third text may be determined by the information source of the third text.

[0192] The authenticity of the text content of the third text is uncertain, and it may be true or fabricated by the first text generation model. The text content of the reference text of the third text is true.

[0193] When the second text is generated based on the first text, that is, the second text includes a second sub-text, if the first response text and the second response text are the same, it means that the text content output by the first text generation model is accurate, that is, the first text generation model did not produce an AI hallucination when processing the first text; if the first response text and the second response text are different, it means that the text content output by the first text generation model is inaccurate, that is, the first text generation model produced an AI hallucination when processing the first text.

[0194] Considering that the verification result may have a large error when the first text generation model generates text less times, in order to accurately determine the verification result of the first text generation model, the model verification method provided by one or more embodiments of the present application can be used for a preset number of first texts, and then the number of times the first text generation model generates AI hallucinations when processing the first text is counted, and the frequency of the first text generation model generating AI hallucinations is determined based on the number and the preset number. If the frequency is greater than or equal to the preset frequency threshold, it can be determined that the verification result of the first text generation model is verified; if the frequency is less than the preset frequency threshold, it can be determined that the verification result of the first text generation model is verified.

[0195] If the verification result of the first text generation model is that the verification is passed, the first text generation model can be put into use. If the verification result of the first text generation model is that the verification is not passed, the first text generation model can continue to be trained for the next round of model to reduce the frequency of the first text generation model generating AI hallucinations.

[0196] In a specific implementation, a verification result of a first text generation model is determined based on a first response text and a second response text, including: extracting keyword information from the second response text; performing query processing in a pre-stored data set based on the keyword information to obtain a target information source of a third text; and determining a verification result of the first text generation model based on a comparison result between the third text and the target information source.

[0197] The keyword information may include index information, address information, or any other data that can be used to locate an information source.

[0198] Extracting keyword information from the second response text may be a preconfigured keyword set, where the keyword set includes multiple preset keywords. When it is detected that the second response text includes any one of the preset keywords in the keyword set, the preset keyword is extracted from the second response text as keyword information.

[0199] The pre-stored data set may be a collection of web pages, a document, a database, or any data storage area storing one or more characteristic data, that is, the aforementioned preset storage area.

[0200] The target information source refers to a sub-region in the preset storage region where feature data of the third text description is stored.

[0201] The keyword information may include index information, such as data table name and column name.

[0202] According to the keyword information, query processing is performed in the pre-stored data set to obtain the target information source of the third text. It can be performed according to the data table name and column name in the database to obtain the target column of the target data table storing the characteristic data described by the third text.

[0203] The keyword information may include address information, such as a web page address and a segment number.

[0204] According to the keyword information, query processing is performed in the pre-stored data set to obtain the target information source of the third text. It can be based on the web page address and segment number, query processing is performed in the web page set to obtain the target segment of the target web page that stores the characteristic data described by the third text.

[0205] The keyword information may include description information, for example, the second line of the historical conversation on the first date.

[0206] According to the keyword information, query processing is performed in the pre-stored data set to obtain the target information source of the third text. The data storage area corresponding to the third text can be determined according to the description information, and query processing is performed in the data storage area to obtain the target text that records the characteristic data described by the third text.

[0207] In a specific implementation, the verification result of the first text generation model is determined based on the comparison result between the third text and the target information source, including: obtaining a reference text of the third text based on the target information source of the third text; if the third text is different from the reference text, determining that the first text generation model generates an abnormal text once; if the third text is the same as the reference text, determining that the first text generation model does not generate abnormal text; when the number of first response texts reaches a first number, counting the number of times the first text generation model generates abnormal texts to obtain a second number; if the ratio of the second number to the first number is greater than a preset ratio threshold, determining that the verification result of the first text generation model is verification failure; if the ratio of the second number to the first number is less than or equal to the preset ratio threshold, determining that the verification result of the first text generation model is verification pass.

[0208] According to the target information source of the third text, a reference text of the third text is obtained.

[0209] Exemplarily, the target information source of the third text may include a target column of a target data table storing characteristic data describing the third text. According to the target information source of the third text, obtaining the reference text of the third text may be by reading the text data stored in the target column of the target data table to obtain the reference text.

[0210] The target information source of the third text may also include a target segment of a target web page storing characteristic data describing the third text. According to the target information source of the third text, the reference text of the third text is obtained by reading the text data stored in the target segment of the target web page to obtain the reference text.

[0211] If the third text is different from the reference text, it is determined that the first text generation model generates an abnormal text once; if the third text is the same as the reference text, it is determined that the first text generation model does not generate an abnormal text.

[0212] The first text generation model generates an abnormal text once, which means that the first response text generated by the first text generation model contains a part of untrue content. In this case, it can be regarded as the first text generation model generates an AI hallucination.

[0213] In addition, for the convenience of statistics, when a first text includes multiple untrue local contents, it can be regarded as that the first text generation model generates an AI hallucination when processing the first text.

[0214] The first text generation model does not generate abnormal text, which means that the text content of the first response text generated by the first text generation model is authentic. In this case, it can be regarded that the first text generation model does not generate AI hallucinations.

[0215] The first number may be a custom-configured number of first texts processed during the process of validating the first text generation model.

[0216] The second number refers to the number of times the first text generation model generates abnormal texts until the first text generation model completes processing of the first number of first texts.

[0217] When the number of first response texts reaches a first number, the number of times the first text generation model generates abnormal texts is counted to obtain a second number.

[0218] If the ratio of the second number to the first number is greater than the preset ratio threshold, the verification result of the first text generation model is determined to be verification failure; if the ratio of the second number to the first number is less than or equal to the preset ratio threshold, the verification result of the first text generation model is determined to be verification passed.

[0219] In a specific implementation, a verification result of a first text generation model is determined based on a first response text and a second response text, including: generating an evaluation score of a third text based on the second response text; determining a weight parameter of the third text based on a data type of the third text; generating a comprehensive verification score of the first text generation model based on the evaluation score of the third text and the weight parameter of the third text; if the comprehensive verification score is greater than a preset score threshold, determining that the verification result of the first text generation model is verification passed; if the comprehensive verification score is less than or equal to the preset score threshold, determining that the verification result of the first text generation model is verification failed.

[0220] An evaluation score for the third text is generated based on the second response text. This may be by determining a target information source for the third text based on the second response text, querying a reference text for the third text based on the target information source, performing similarity calculation processing on the third text and the reference text to obtain a similarity calculation result, and generating an evaluation score for the third text based on the similarity calculation result.

[0221] For each third text, the data type of the third text may be one of a preset data type set.

[0222] For example, the preset data type set includes: amount, repayment time, historical events, and so on.

[0223] The data type of the third text 1 "350 yuan" is the amount of money. The data type of the third text 2 "mentioned yesterday" is "historical event". The data type of the third text 3 "X month Y day" is the repayment time.

[0224] According to the data type of the third text, a weight parameter of the third text is determined. There is a corresponding relationship between the data type and the weight parameter, and the corresponding relationship can be determined by the importance of the hallucination starting point of the AI ​​hallucination.

[0225] The starting point of hallucination refers to the event element that causes the AI ​​hallucination. The starting point of hallucination may be a person, time, place, state, emotion, cause, result, etc.

[0226] The importance of each illusion starting point can be pre-configured. For example, in an event, the importance of each illusion starting point is ranked as follows:

[0227] Person>Time / Place>Result>Cause>State / Emotion.

[0228] For each third text, the hallucination starting point of the third text can be determined according to the data type of the third text. Different hallucination starting points are respectively configured with corresponding weight parameters, so the weight parameter of the third text can be determined according to the hallucination starting point of the third text.

[0229] Generating a comprehensive evaluation score of the first text generation model according to the evaluation score of the third text and the weight parameter of the third text can be achieved in the following manner:

[0230] The number of first response texts that need to be generated before determining the verification result of the first text generation model is predetermined. For example, after the first text generation model generates 20 first response texts, the conversation ends.

[0231] For each first response text, the evaluation score of the third text and the weight parameter of the third text in the first response text are determined by executing the model verification method provided by one or more embodiments of the present application.

[0232] A weighted summation process is performed based on the evaluation score of the third text and the weight parameter of the third text to obtain a comprehensive evaluation score of the first text generation model. If the comprehensive verification score is greater than a preset score threshold, the verification result of the first text generation model is determined to be passed; if the comprehensive verification score is less than or equal to the preset score threshold, the verification result of the first text generation model is determined to be failed.

[0233] In a specific implementation, the verification result of the first text generation model includes the verification score of the first text generation model; the number of first text generation models is multiple; the model verification method also includes: sorting each first text generation model according to the evaluation score of each first text generation model to obtain a sorting result; according to the sorting result, selecting a preset number of first text generation models as target text generation models.

[0234] The number of the first text generation models can be multiple.

[0235] After determining the verification result of each first text generation model, an evaluation score of each first text generation model may be obtained.

[0236] According to the evaluation score of each first text generation model, each first text generation model is sorted to obtain a sorting result.

[0237] Exemplarily, the higher the evaluation score of the first text generation model, the lower the frequency of the first text generation model producing AI hallucinations, the more reliable the model is, and the higher its ranking result is.

[0238] According to the sorting results, a preset number of first text generation models are selected as target text generation models. One or more first text generation models with the highest evaluation scores are selected as target text generation models according to the sorting results. The target text generation models can be put into use.

[0239] Through the model verification method provided in the embodiment of the present application, a verification result that objectively evaluates the accuracy of a first text generation model can be obtained for a first text generation model, and the same first text can be used to simultaneously evaluate multiple first text generation models to provide relative evaluation results of the multiple first text generation models.

[0240] For example, for the same first text, after evaluating the first text generation model 1, the comprehensive evaluation score of the first text generation model 1 is K1, and after evaluating the first text generation model 2, the comprehensive evaluation score of the first text generation model 1 is K2, and both K1 and K2 are real numbers greater than 0. By comparing K1 and K2, the relative evaluation results of the first text generation model 1 and the first text generation model 2 can be determined.

[0241] In such Figure 2In the embodiment shown, first, the first text is input into the first text generation model for text generation to obtain the first answer text of the first text; then, the second text is generated according to the first answer text or the first text; then, the second text is input into the first text generation model for text generation to obtain the second answer text of the second text; finally, the verification result of the first text generation model is determined according to the first answer text and the second answer text. In this way, considering that the essence of the text generation of the first text generation model is to complete the input text, rather than data query, the text content of the first answer text obtained by the first text generation model according to the input first text may be true and reliable, or it may be fabricated by the first text generation model. By inputting the second text generated according to the first answer text or the first text into the first text generation model for text generation to obtain the second answer text, the first answer text and the second answer text can mutually verify the authenticity of each other's text content. For any text generation model, the more frequently the model outputs wrong content, the lower its accuracy and the more unreliable the model is; the rarer the model outputs wrong content, the higher its accuracy and the more reliable the model is. Therefore, by mutually verifying the authenticity of the other party's text content through the first answer text and the second answer text, it is possible to verify whether the first text generation model is reliable. In addition, in the entire verification process, the second text is not manually written, but generated according to the first answer text or the first text, which reduces manual participation and improves the efficiency of dialogue, thereby indirectly improving the model verification efficiency. Therefore, the model verification method provided in the embodiment of the present application can efficiently verify whether the first text generation model is reliable.

[0242] The following takes the application of a model verification method provided in this embodiment in a consulting scenario as an example. Figure 3 The model verification method provided in this embodiment is further explained. Figure 3 A processing flow chart of another model verification method provided in an embodiment of the present application.

[0243] like Figure 3 As shown, in step S302, a large model 1 to be evaluated is initialized.

[0244] The large model 1 can refer to the aforementioned first text generation model.

[0245] Initialization can be to perform model training on the large model 1 to be evaluated through the first training sample of the target domain.

[0246] For example, in a consulting scenario, training samples in the target domain can be generated based on the conversation text between the agent and the customer. After model training, the large model 1 can simulate the agent to perform text generation processing.

[0247] Step S304, initializing another large model 2.

[0248] The large model 2 may include the aforementioned second text generation model and the third text generation model.

[0249] Initialization may be to train the large model 2 through the second training sample of the target domain. After the model training, the large model 2 may simulate the customer to perform text generation processing.

[0250] Step S306, randomly generate a speaking order so that large model 1 and large model 2 can have a conversation.

[0251] Randomly determine which of the large model 1 and the large model 2 will start the conversation first.

[0252] For example, two robots are divided into two roles, robot A includes large model 1, and robot B includes large model 2. Robot A sends a text text1, inputs text1 into robot B, and robot B outputs text2 as a reply to text1; robot B inputs text2 into robot A, and robot A outputs text3 as a reply to text2, and the cycle repeats to form a conversation.

[0253] Step S308: If the text generated by big model 1 includes entities and expressions that may cause hallucinations, such as historical events and specific amounts, big model 2 will inquire about the source of the information.

[0254] This step may refer to the corresponding description of the aforementioned steps S204 to S206.

[0255] In step S310, the large model 1 responds to the information source. If the response does not conform to the actual situation, it is determined that a hallucination occurs and the weight parameter of the hallucination is determined.

[0256] Step S312, after blocking part of the information in the prompt information or part of the events in the historical conversation, re-inquire the large model 1.

[0257] This step may refer to the corresponding description parts of the aforementioned steps S204 to S206.

[0258] Step S314: after one or more conversations are completed, the large model 1 is evaluated.

[0259] Since the technical concept is the same, the description in this embodiment is relatively simple, and the relevant parts may refer to the corresponding description of the method embodiment provided above.

[0260] An embodiment of a model verification device provided in this specification is as follows:

[0261] In the above-mentioned embodiment, a model verification method is provided. Based on the same technical concept, the embodiment of the present application also provides a model verification device, which is described below in conjunction with the accompanying drawings.

[0262] Figure 4 A schematic diagram of a model verification device provided in an embodiment of the present application. This embodiment provides a model verification device 400, including:

[0263] A generating unit 402 is used to input a first text into a first text generating model to perform text generating processing, so as to obtain a first response text of the first text;

[0264] The generating unit 402 is further configured to generate a second text according to the first response text or the first text;

[0265] The generating unit 402 is further configured to input the second text into the first text generating model for text generating processing to obtain a second response text of the second text;

[0266] The determination unit 404 is used to determine the verification result of the first text generation model according to the first response text and the second response text.

[0267] Optionally, the second text includes a first subtext; when the generating unit 402 generates the second text according to the first response text, the generating unit 402 performs the following steps:

[0268] Extracting information from the first response text to obtain an information extraction result;

[0269] When the information extraction result includes a third text indicating that the first response text has an abnormal risk, the third text is input into a second text generation model for text generation to obtain the first sub-text; the first sub-text is used to question the information source of the third text.

[0270] Optionally, the second text includes a second subtext; when the generating unit 402 generates the second text according to the first text, the generating unit 402 performs the following steps:

[0271] Determining a first character to be blocked in the first text;

[0272] The first character is replaced in the first text by using the preset shielding character to obtain the second sub-text.

[0273] Optionally, the model verification device 400 further includes:

[0274] A receiving unit, configured to receive a fourth text; the fourth text is generated by using the third text generation model;

[0275] The determining unit 404 is further configured to determine a prompt text corresponding to the fourth text;

[0276] A concatenation unit is used to concatenate the fourth text with the prompt text to obtain the first text.

[0277] Optionally, when determining the verification result of the first text generation model according to the first response text and the second response text, the determining unit 404 performs the following steps:

[0278] extracting keyword information from the second response text;

[0279] According to the keyword information, query processing is performed in a pre-stored data set to obtain a target information source of the third text;

[0280] A verification result of the first text generation model is determined according to a comparison result between the third text and the target information source.

[0281] Optionally, when determining the verification result of the first text generation model according to the comparison result between the third text and the target information source, the determining unit 404 performs the following steps:

[0282] Acquiring a reference text of the third text according to a target information source of the third text;

[0283] If the third text is different from the reference text, determining that the first text generation model generates an abnormal text once;

[0284] If the third text is the same as the reference text, determining that the first text generation model does not generate the abnormal text;

[0285] When the number of the first response texts reaches a first number, counting the number of times the first text generation model generates the abnormal texts to obtain a second number;

[0286] If the ratio of the second number to the first number is greater than a preset ratio threshold, determining that the verification result of the first text generation model is verification failure;

[0287] If the ratio of the second number to the first number is less than or equal to a preset ratio threshold, it is determined that the verification result of the first text generation model is passed.

[0288] Optionally, when determining the verification result of the first text generation model according to the first response text and the second response text, the determining unit 404 performs the following steps:

[0289] generating an evaluation score for the third text according to the second response text;

[0290] Determining a weight parameter of the third text according to the data type of the third text;

[0291] generating a comprehensive verification score of the first text generation model according to the evaluation score of the third text and the weight parameter of the third text;

[0292] If the comprehensive verification score is greater than the preset score threshold, determining that the verification result of the first text generation model is verified passed;

[0293] If the comprehensive verification score is less than or equal to the preset score threshold, it is determined that the verification result of the first text generation model is verification failure.

[0294] Optionally, the model verification device 400 further includes:

[0295] a classification unit, configured to classify the first response text to obtain a classification result of the first response text;

[0296] an extraction unit, configured to extract information from the first response text to obtain an information extraction result;

[0297] The determining unit 404 is further configured to determine that an information extraction anomaly exists if the classification result indicates that the first response text does not include a text with an abnormal risk, and the information extraction result includes a third text with an abnormal risk;

[0298] The determining unit 404 is further configured to determine that an information extraction anomaly exists if the classification result indicates that the first response text includes text with an abnormal risk and the information extraction result is empty;

[0299] A marking unit is used to mark the first response text with information extraction anomaly and the first text corresponding to the first response text.

[0300] Optionally, the verification result of the first text generation model includes a verification score of the first text generation model; the number of the first text generation models is multiple; the model verification device 400 further includes:

[0301] A sorting unit, configured to sort each of the first text generation models according to the verification score of each of the first text generation models to obtain a sorting result;

[0302] A selection unit is used to select a preset number of first text generation models as target text generation models according to the sorting result.

[0303] In an embodiment of the present application, the model verification device includes: a generation unit, which is used to input the first text into the first text generation model for text generation processing to obtain a first response text of the first text; the generation unit is also used to generate a second text according to the first response text or the first text; the generation unit is also used to input the second text into the first text generation model for text generation processing to obtain a second response text of the second text; and a determination unit is used to determine the verification result of the first text generation model according to the first response text and the second response text. In this way, considering that the essence of the text generation of the first text generation model is to complete the input text, rather than data query, the text content of the first response text obtained by the first text generation model according to the input first text may be true and reliable, or it may be fabricated by the first text generation model. By inputting the second text generated according to the first response text or the first text into the first text generation model for text generation to obtain the second response text, the first response text and the second response text can mutually verify the authenticity of each other's text content. For any text generation model, the more frequently the model outputs wrong content, the lower its accuracy and the more unreliable the model is. The rarer the model outputs wrong content, the higher its accuracy and the more reliable the model is. Therefore, by mutually verifying the authenticity of the other party's text content through the first answer text and the second answer text, it is possible to verify whether the first text generation model is reliable. In addition, in the entire verification process, the second text is not manually written, but generated according to the first answer text or the first text, which reduces manual participation and improves the efficiency of dialogue, thereby indirectly improving the model verification efficiency. Therefore, the model verification method provided in the embodiment of the present application can efficiently verify whether the first text generation model is reliable.

[0304] Corresponding to the model verification method described above, based on the same technical concept, an embodiment of the present application further provides an electronic device, which is used to execute the model verification method provided above. Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0305] like Figure 5As shown, the electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 501 and memory 502, and the memory 502 may store one or more storage applications or data. Among them, the memory 502 can be a temporary storage or a permanent storage. The application stored in the memory 502 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 501 can be configured to communicate with the memory 502 to execute a series of computer executable instructions in the memory 502 on the electronic device. The electronic device may also include one or more power supplies 503, one or more wired or wireless network interfaces 504, one or more input / output interfaces 505, one or more keyboards 506, etc.

[0306] In a specific embodiment, the electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for the electronic device, and the one or more programs configured to be executed by one or more processors include the following computer executable instructions:

[0307] Inputting the first text into a first text generation model to generate text, thereby obtaining a first response text to the first text;

[0308] Generate a second text according to the first response text or the first text;

[0309] Inputting the second text into the first text generation model to generate text, thereby obtaining a second response text for the second text;

[0310] A verification result of the first text generation model is determined based on the first response text and the second response text.

[0311] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions. When the computer-executable instructions are executed by a processor, the following process is implemented:

[0312] Inputting the first text into a first text generation model to generate text, thereby obtaining a first response text to the first text;

[0313] Generate a second text according to the first response text or the first text;

[0314] Inputting the second text into the first text generation model to generate text, thereby obtaining a second response text for the second text;

[0315] A verification result of the first text generation model is determined based on the first response text and the second response text.

[0316] It should be noted that the embodiment of the computer-readable storage medium in this specification and the embodiment of the model verification method in this specification are based on the same inventive concept, so the specific implementation of this embodiment can refer to the implementation of the aforementioned corresponding method, and the repeated parts will not be repeated.

[0317] Another embodiment of the present disclosure further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the following process is implemented:

[0318] Inputting the first text into a first text generation model to generate text, thereby obtaining a first response text to the first text;

[0319] Generate a second text according to the first response text or the first text;

[0320] Inputting the second text into the first text generation model to generate text, thereby obtaining a second response text for the second text;

[0321] A verification result of the first text generation model is determined based on the first response text and the second response text.

[0322] The computer program product in the embodiment of the present disclosure can implement each process of the above-mentioned model verification method embodiment and achieve the same effect and function, which will not be repeated here.

[0323] In the embodiment of the present application, first, the first text is input into the first text generation model for text generation to obtain the first answer text of the first text; then, according to the first answer text or the first text, the second text is generated; then, the second text is input into the first text generation model for text generation to obtain the second answer text of the second text; finally, according to the first answer text and the second answer text, the verification result of the first text generation model is determined. In this way, considering that the essence of the text generation of the first text generation model is to complete the input text, rather than data query, therefore, the text content of the first answer text obtained by the first text generation model according to the input first text for text generation may be true and reliable, or it may be fabricated by the first text generation model, by inputting the second text generated according to the first answer text or the first text into the first text generation model for text generation to obtain the second answer text, and then, the first answer text and the second answer text can mutually verify the authenticity of each other's text content. For any text generation model, the more frequently the model outputs wrong content, the lower its accuracy, the more unreliable the model, the rarer the model outputs wrong content, the higher its accuracy, and the more reliable the model. Therefore, by mutually verifying the authenticity of the other party's text content through the first answer text and the second answer text, it is possible to verify whether the first text generation model is reliable. In addition, in the entire verification process, the second text is not manually written, but generated according to the first answer text or the first text, which reduces manual participation and improves the efficiency of dialogue, thereby indirectly improving the model verification efficiency. Therefore, the model verification method provided in the embodiment of the present application can efficiently verify whether the first text generation model is reliable.

[0324] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0325] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0326] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable electronic device to produce a machine, so that the instructions executed by the processor of the computer or other programmable electronic device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0327] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable electronic device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0328] These computer program instructions may also be loaded onto a computer or other programmable electronic device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0329] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0330] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0331] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0332] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0333] Embodiments of the present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0334] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0335] The above description is only an embodiment of this document and is not intended to limit this document. For those skilled in the art, this document may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this document should be included in the scope of the claims of this document.

Claims

1. A model validation method, It is characterized in that include: Inputting the first text into a first text generation model to generate text, thereby obtaining a first response text to the first text; Generate a second text according to the first response text or the first text; Inputting the second text into the first text generation model to generate text, thereby obtaining a second response text for the second text; A verification result of the first text generation model is determined based on the first response text and the second response text.

2. The method according to claim 1, It is characterized in that The second text includes a first subtext; Generating the second text according to the first response text includes: Extracting information from the first response text to obtain an information extraction result; When the information extraction result includes a third text indicating that the first response text has an abnormal risk, the third text is input into a second text generation model for text generation to obtain the first sub-text; the first sub-text is used to question the information source of the third text.

3. The method according to claim 1, It is characterized in that The second text includes a second subtext; and generating the second text according to the first text includes: Determining a first character to be blocked in the first text; The first character is replaced in the first text by using the preset shielding character to obtain the second sub-text.

4. The method according to claim 1, It is characterized in that Before inputting the first text into the first text generation model to generate text and obtaining a first response text for the first text, the method further includes: receiving a fourth text; wherein the fourth text is generated by using the third text generation model; Determine a prompt text corresponding to the fourth text; The first text is obtained by concatenating the fourth text with the prompt text.

5. The method according to claim 2, It is characterized in that The step of determining the verification result of the first text generation model according to the first response text and the second response text includes: extracting keyword information from the second response text; According to the keyword information, query processing is performed in a pre-stored data set to obtain a target information source of the third text; A verification result of the first text generation model is determined according to a comparison result between the third text and the target information source.

6. The method according to claim 5, It is characterized in that The step of determining the verification result of the first text generation model according to the comparison result between the third text and the target information source includes: Acquiring a reference text of the third text according to a target information source of the third text; If the third text is different from the reference text, determining that the first text generation model generates an abnormal text once; If the third text is the same as the reference text, determining that the first text generation model does not generate the abnormal text; When the number of the first response texts reaches a first number, counting the number of times the first text generation model generates the abnormal texts to obtain a second number; If the ratio of the second number to the first number is greater than a preset ratio threshold, determining that the verification result of the first text generation model is verification failure; If the ratio of the second number to the first number is less than or equal to a preset ratio threshold, it is determined that the verification result of the first text generation model is passed.

7. The method according to claim 2, It is characterized in that The step of determining the verification result of the first text generation model according to the first response text and the second response text includes: generating an evaluation score for the third text according to the second response text; Determining a weight parameter of the third text according to the data type of the third text; generating a comprehensive verification score of the first text generation model according to the evaluation score of the third text and the weight parameter of the third text; If the comprehensive verification score is greater than the preset score threshold, determining that the verification result of the first text generation model is verified passed; If the comprehensive verification score is less than or equal to the preset score threshold, it is determined that the verification result of the first text generation model has failed the verification.

8. The method according to claim 1, It is characterized in that The method further comprises: Classifying the first response text to obtain a classification result of the first response text; Extracting information from the first response text to obtain an information extraction result; If the classification result indicates that the first response text does not include text with an abnormal risk, and the information extraction result includes a third text with an abnormal risk, it is determined that there is an information extraction abnormality; If the classification result indicates that the first response text includes text with an abnormal risk, and the information extraction result is empty, it is determined that there is an information extraction abnormality; The first response text that marks the existence of information extraction anomaly and the first text corresponding to the first response text.

9. The method according to claim 1, It is characterized in that The verification result of the first text generation model includes a verification score of the first text generation model; the number of the first text generation models is multiple; and the method further includes: According to the verification score of each of the first text generation models, each of the first text generation models is sorted to obtain a sorting result; According to the sorting result, a preset number of first text generation models are selected as target text generation models.

10. A model verification device, It is characterized in that include: A generating unit, configured to input a first text into a first text generating model for text generating processing, and obtain a first response text of the first text; The generating unit is further configured to generate a second text according to the first response text or the first text; The generating unit is further configured to input the second text into the first text generating model for text generating processing to obtain a second response text of the second text; A determination unit is used to determine a verification result of the first text generation model according to the first response text and the second response text.

11. An electronic device, It is characterized in that include: processor; and a memory configured to store computer executable instructions, which, when executed, cause the processor to perform the model validation method according to any one of claims 1 to 9.

12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the model verification method according to any one of claims 1 to 9 is implemented.

13. A computer program product, It is characterized in that The invention comprises a computer program, which implements the model verification method according to any one of claims 1 to 9 when being executed by a processor.

Images