Illusion detection method of large model, computer, storage medium and program product

Abstract

The invention discloses a hallusion detection method for a large model, a computer, a storage medium and a program product, and the method comprises the steps: inputting a first text into the large model, and obtaining a replied second text; calculating a first correlation score of the first text and each second text and a second correlation score of each second text; and based on the mapping relationship, determining a first cue word corresponding to the first correlation score and a second cue word corresponding to the second correlation score, and inputting features of the first cue word and the second cue word into the first model to obtain a first probability used for representing an illusion detection result of the large model. The relevance between the question and the reply text and the relevance between the reply texts are quantified to obtain a relevance score, then the features of the cue word are generated based on the relevance score and the mapping relation, and the illusion probability is predicted through the model. The hallucination detection of a large model is realized from the perspective of relevance between reply texts and between the reply texts and questions, and the hallucination detection accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a large-model hallucination detection method, a computer, a storage medium, and a program product. Background Art

[0002] Big model hallucination refers to the generation of erroneous, fictitious or contradictory content by a model when it has no factual basis or deviates from reference materials. As the application of big models expands, its concealment increases and may cause serious consequences in related fields, thus giving rise to the demand for big model hallucination detection.

[0003] To detect hallucinations in large models, the model's output is typically compared to reference data. If the model's output differs from the reference information, hallucinations are present. However, this approach relies heavily on reference data, making hallucination detection less accurate when the reference data itself is erroneous, outdated, or incomplete. Summary of the Invention

[0004] The embodiments of the present application provide a large-model hallucination detection method, computer, storage medium, and program product. Without relying on reference materials, the method generates text features by combining the correlation between questions and reply texts and the correlation between reply texts, and uses the model to predict the probability of hallucination. Hallucinations are detected from the perspective of the correlation between reply texts and the correlation between reply texts and questions, thereby improving the accuracy of hallucination detection.

[0005] In a first aspect, an embodiment of the present application provides a large model hallucination detection method, the method comprising: Inputting a first text into a large model to obtain a plurality of second texts that respond to the first text; Calculating, based on the first text and the second text, a first relevance score between the first text and each of the plurality of second texts and a second relevance score between each of the second texts; Determining, based on a mapping relationship between prompt words and relevance scores, a first prompt word corresponding to each first relevance score and a second prompt word corresponding to each second relevance score; concatenating the first prompt word corresponding to each of the first relevance scores and the second prompt word corresponding to each of the second relevance scores to obtain a third prompt word corresponding to each of the second texts; Performing feature extraction on the third prompt word corresponding to each second text to obtain features of the third prompt word corresponding to each second text; The features of the third prompt word corresponding to each second text are input into the first model to obtain a first probability of each second text, where the first probability is used to represent the hallucination detection result of the large model.

[0006] In a second aspect, an embodiment of the present application provides a large-scale hallucination detection device, comprising: An acquisition module, configured to input a first text into a large model and obtain a plurality of second texts that respond to the first text; a processing module, configured to calculate, based on the first text and the second text, a first relevance score between the first text and each of the plurality of second texts and a second relevance score between each of the second texts; and determining, based on a mapping relationship between prompt words and relevance scores, a first prompt word corresponding to each first relevance score and a second prompt word corresponding to each second relevance score; and concatenating the first prompt word corresponding to each of the first relevance scores and the second prompt word corresponding to each of the second relevance scores to obtain a third prompt word corresponding to each of the second texts; and extracting features of the third prompt word corresponding to each second text to obtain features of the third prompt word corresponding to each second text; And it is used to input the feature of the third prompt word corresponding to each second text into the first model to obtain the first probability of each second text, and the first probability is used to represent the hallucination detection result of the large model.

[0007] In a third aspect, an embodiment of the present application provides a computer, including: A memory, a processor, and an executable program code stored in the memory and executable on the processor, wherein the executable program code is configured to implement part or all of the steps described in any method in the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which is stored a prosumer energy optimization program that takes into account the life loss of energy storage. When the prosumer energy optimization program that takes into account the life loss of energy storage is executed by a processor, some or all of the steps described in any method in the first aspect are implemented.

[0009] In a fifth aspect, embodiments of the present application provide a computer program product, wherein the computer program product includes a computer program operable to cause a computer to perform some or all of the steps described in any method of the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0010] In an embodiment of the present application, a first text is first input into a large model to obtain multiple second texts that reply to the first text; then, based on the first text and the second text, a first correlation score between the first text and each of the multiple second texts and a second correlation score for each second text are calculated; based on the mapping relationship between the prompt word and the correlation score, the first prompt word corresponding to each first correlation score and the second prompt word corresponding to each second correlation score are determined; the first prompt word corresponding to each first correlation score and the second prompt word corresponding to each second correlation score are concatenated to obtain a third prompt word corresponding to each second text; then, feature extraction is performed on the third prompt word corresponding to each second text to obtain the features of the third prompt word corresponding to each second text; finally, the features of the third prompt word corresponding to each second text are input into the first model to obtain a first probability for each second text, which is used to represent the hallucination detection result of the large model. By combining the correlation between the question and the multiple reply texts output by the model, as well as the correlation between the multiple reply texts, the features of the prompt word are generated, and the first probability is obtained by combining the features of the prompt word and the prediction of the first model, thereby obtaining the hallucination detection result of the large model, realizing hallucination detection from the perspective of the correlation between the reply texts and the correlation between the reply texts and the question, without relying on reference materials, and improving the accuracy of hallucination detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0012] Figure 1 This is a schematic diagram of the architecture of a large-scale hallucination detection system provided in an embodiment of the present application; Figure 2 This is a flow chart of a large model hallucination detection method provided in an embodiment of the present application; Figure 3 This is a flow chart of a method for obtaining a first correlation score and a second correlation score provided in an embodiment of the present application; Figure 4 is a flow chart of a method for obtaining a second relevance score for each second text provided in an embodiment of the present application; Figure 5 is a flow chart of a method for determining a fifth text from multiple second texts provided by an embodiment of the present application; Figure 6 This is a flow chart of a method for training a second model to obtain a first model provided in an embodiment of the present application; Figure 7 This is a schematic structural diagram of a large-scale hallucination detection device provided in an embodiment of the present application; Figure 8It is a structural diagram of a computer provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work should fall within the scope of protection of the present invention.

[0014] The terms "first," "second," and "third," etc. in the specification, claims, and drawings of this application are used to distinguish between different objects, not to describe a particular order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0015] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0016] Big model hallucination refers to the generation of erroneous, fictitious or contradictory content by a model when it has no factual basis or deviates from reference materials. As the application of big models expands, its concealment increases and may cause serious consequences in related fields, thus giving rise to the demand for big model hallucination detection.

[0017] To detect hallucinations in large models, the model's output is typically compared to reference data. If the model's output differs from the reference information, hallucinations are present. However, this approach relies heavily on reference data, making hallucination detection less accurate when the reference data itself is erroneous, outdated, or incomplete.

[0018] In an embodiment of the present application, a large model hallucination detection method, computer, storage medium and program product are provided. First, a first text is input into the large model to obtain multiple second texts that reply to the first text; then, based on the first text and the second text, a first correlation score between the first text and each second text in the multiple second texts and a second correlation score of each second text are calculated; based on the mapping relationship between the prompt word and the correlation score, the first prompt word corresponding to each first correlation score and the second prompt word corresponding to each second correlation score are determined; the first prompt word corresponding to each first correlation score and the second prompt word corresponding to each second correlation score are spliced ​​to obtain a third prompt word corresponding to each second text; then, feature extraction is performed on the third prompt word corresponding to each second text to obtain the feature of the third prompt word corresponding to each second text; finally, the feature of the third prompt word corresponding to each second text is input into the first model to obtain a first probability for each second text, and the first probability is used to represent the hallucination detection result of the large model. By combining the correlation between the question and multiple reply texts output by the model, as well as the correlation between multiple reply texts, the features of the prompt word are generated, and the first probability is obtained by combining the text features and the first model prediction, thereby obtaining the hallucination detection result of the large model. It is possible to detect hallucinations from the perspective of the correlation between the reply texts and the correlation between the reply texts and the question, without relying on reference materials, thereby improving the accuracy of hallucination detection.

[0019] The large model hallucination detection method, computer, storage medium and program product provided in the embodiments of the present application can be applied to Figure 1 For a larger model of the hallucination detection system, see Figure 1 , Figure 1 This is a schematic diagram of the architecture of a large-model hallucination detection system provided in an embodiment of the present application. The large-model hallucination detection system 100 includes a terminal 101 and a server 102. The terminal 101 can communicate with the server 102 through a network. The terminal 101 refers to a device used by the user, such as a smart phone, a computer, etc. In this solution, the terminal 101 provides an interface for the user to interact with the large-model hallucination detection system 100. Through the terminal 101, the user can interact with the large-model hallucination detection system 100, receive the hallucination detection results sent from the server 102, and display them on the user interface. The user can understand the first probability of the model output through the terminal 101 and set the relevant parameters required in the hallucination detection, such as the method for obtaining the first numerical value, and the weight used when calculating the weighted sum of the first relevance score and the second relevance score of each second text.

[0020] Server 102 refers to a remote computer used to process large amounts of computing tasks and store data. In this solution, server 102 is responsible for inputting a first text into a large model to obtain multiple second texts that respond to the first text; then, based on the first text and the second text, a first relevance score between the first text and each of the multiple second texts and a second relevance score for each second text are calculated; based on the mapping relationship between the prompt word and the relevance score, a first prompt word corresponding to each first relevance score and a second prompt word corresponding to each second relevance score are determined; the first prompt word corresponding to each first relevance score and the second prompt word corresponding to each second relevance score are concatenated to obtain a third prompt word corresponding to each second text; then, feature extraction is performed on the third prompt word corresponding to each second text to obtain features of the third prompt word corresponding to each second text; finally, the features of the third prompt word corresponding to each second text are input into the first model to obtain a first probability for each second text, and the first probability is used to represent the hallucination detection result of the large model.

[0021] Based on this, this application provides a large model hallucination detection method, computer, storage medium and program product, please refer to Figure 2 , Figure 2 This is a flow chart of a large model hallucination detection method provided in an embodiment of the present application. The present application is described in detail below in conjunction with the accompanying drawings.

[0022] S201: Input a first text into a large model to obtain a plurality of second texts that respond to the first text.

[0023] Among them, the execution subject of this method can be Figure 1 The server 102 in the hallucination detection system 100 for medium and large models.

[0024] The large model is the one that needs to be used for hallucination detection. In this solution, the large model is a large language model (LLM), a natural language processing model with a large number of parameters. This model typically requires extensive computing resources and training data to handle a variety of complex natural language tasks, including language understanding, generation, and translation.

[0025] The first text is a complete question in the form of an interrogative sentence, containing interrogative semantics. For example, the first text could be "What color is an apple?" After inputting the first text into the large model, the large model can be called multiple times or with different generation parameters to generate at least two responses to the same question. These responses are the second texts. There is no limit on the number of second texts the model can output.

[0026] For example, the first text is "What is a red fruit?", and the first model outputs multiple second texts including second text 1: "Apples are red fruits," second text 2: "Strawberries are red fruits," and second text 3: "Tomatoes are red fruits." Subsequently, hallucination detection is performed on the first text and the three second texts output by the first model.

[0027] S202 : Calculate, based on the first text and the second text, a first relevance score between the first text and each second text in the plurality of second texts and a second relevance score between each second text.

[0028] The first relevance score is obtained by comparing the first and second texts. For each second text, a first relevance score can be obtained for the second text and the first text. The first relevance score can be used to evaluate the entailment relationship (necessary reasoning relationship), neutral relationship, and conflict relationship between the first and second texts. Specifically, the entailment relationship indicates whether a conclusion (here, the second text) can be necessarily inferred from another premise (here, the first text); the neutral relationship indicates that the conclusion (here, the second text) is unrelated to the premise (here, the first text); and the conflict relationship indicates that the conclusion (here, the second text) directly contradicts the premise (here, the first text).

[0029] Among them, a large model can be used to obtain a first relevance score between the first text and each second text in multiple second texts based on the input first text and the second text. Specifically, in addition to being a large language model, the large model can also be an NLP large model (such as GPT-4, LLaMA2), a text implication model (such as RoBERTa-wwm-ext, ESIM, BERT-ST) and a multilingual model (such as Google PaLM 2, Meta SeamlessM4T), etc.

[0030] In addition to using large models to obtain the first relevance score between questions and responses, you can also use pre-trained model fine-tuning, knowledge graph reasoning, and traditional machine learning methods.

[0031] Specifically, pre-trained models are fine-tuned using open-source pre-trained models, such as BERT. These models are pre-trained using public NLI datasets (such as SNLI, MNLI, and CNLI). They are then fine-tuned on a custom question-answer dataset, outputting a three-category classification result (implied / neutral / conflicting). Finally, the question and answer are concatenated into the form "[question][SEP][reply]", which is then fed into the model to obtain sentence embeddings. The model's fully connected layer then predicts the relevance score.

[0032] Specifically, the knowledge graph reasoning method first extracts domain knowledge and stores it as triples (subject, predicate, object), for example, (cat, belongs to, mammal). Logical reasoning is then performed. Implication relationships: If the response can be directly derived from the knowledge graph (such as "cats are mammals"), the implication relationship score is high. Conflict: If the response contradicts the knowledge graph (such as "cats are reptiles"), the conflict relationship score is high. Neutral relationships: If there is no relevant information in the knowledge graph (such as "Do cats like blue"), the neutral relationship score is high. Finally, the relevance score is obtained based on the reasoning results.

[0033] Specifically, traditional machine learning approaches rely on manually designed semantic features combined with classification algorithms to determine logical relationships. They first acquire lexical features, syntactic features (such as subject-verb-object structure matching and dependency syntactic relationships), and logical features (such as quantifiers and modal terms) from questions and responses. Then, algorithms such as support vector machines (SVMs) and random forests are used to train models based on labeled implication, neutrality, and conflict datasets. Relevance scores are then generated based on the trained model and the input questions and responses.

[0034] Unlike the first relevance score, the second relevance score is obtained based on different second texts. For each second text, a second relevance score can be obtained for that second text and other second texts. The second relevance score can be used to assess the implication, neutrality, and conflict between different second texts.

[0035] Among them, a large model can be used to obtain second relevance scores of different second texts based on the input second text. Specifically, in addition to being a large language model, the large model can also be an NLP large model (such as GPT-4, LLaMA2), a text entailment model (such as RoBERTa-wwm-ext, ESIM, BERT-ST) and a multilingual model (such as Google PaLM 2, MetaSeamlessM4T), etc.

[0036] In addition to using a large model to obtain the second relevance score of different replies (i.e., the second text), you can also use pre-trained model fine-tuning, knowledge graph reasoning, and traditional machine learning methods.

[0037] Specifically, pre-trained models are fine-tuned using open-source pre-trained models, such as BERT. These models are pre-trained using public NLI datasets (e.g., SNLI, MNLI, and CNLI). They are then fine-tuned on a custom reply-reply dataset, outputting a three-category classification result (implied / neutral / conflicting). Finally, the different replies (e.g., reply 1, reply 2) are concatenated into the form "[reply 1] [SEP][reply 2]," which is then fed into the model to obtain sentence embeddings. A fully connected layer is then used to predict the relevance score between reply 1 and reply 2.

[0038] Specifically, the knowledge graph reasoning method first extracts domain knowledge and stores it as triples (subject, predicate, object), such as (cat, belongs to, mammal). Logical reasoning is then performed. Implication relationships: If the response can be directly derived from the knowledge graph (such as "cats are mammals"), the implication relationship score is high. Conflict relationships: If the response contradicts the knowledge graph (such as "cats are reptiles"), the conflict score is high. Neutral relationships: If there is no relevant information in the knowledge graph (such as "Do cats like blue"), the neutral relationship score is high. Finally, the relevance score is obtained based on the reasoning results.

[0039] Specifically, traditional machine learning approaches rely on manually designed semantic features combined with classification algorithms to determine logical relationships. They first capture lexical features, syntactic features (such as subject-verb-object structure matching and dependency syntactic relationships), and logical features (such as quantifiers and modal terms) from different responses. Then, algorithms such as support vector machines (SVMs) and random forests are used to train models based on labeled implication, neutrality, and conflict datasets. Relevance scores are then generated based on the trained model and the different responses input.

[0040] In one possible implementation, see Figure 3 , Figure 3 This is a flow chart of a method for obtaining a first correlation score and a second correlation score provided in an embodiment of the present application. Figure 3 As shown, according to the first text and the second text, calculating a first relevance score between the first text and each second text in the plurality of second texts and a second relevance score for each second text includes the following steps: S301: Input the first text, each second text in the plurality of second texts, and a preset second prompt word into the large model to obtain a first relevance score between the first text output by the large model and each second text in the plurality of second texts.

[0041] The second prompt word can be set or modified by the user at the terminal 101 of the large-scale hallucination detection system 100 , and the first relevance score includes an implication score, a neutrality score, and a conflict score.

[0042] Among them, users can send the preset second prompt word to the selected large model through the API interface, and then obtain the first relevance score output by the large model. For example, the first text is regarded as a question, the second text is regarded as a reply, and the second prompt word is as follows: "You are an expert in evaluating text quality and will receive user questions and AI replies. Your task is to evaluate the implication score, neutrality score and conflict score of the AI ​​reply and the question. You need to first analyze and then strictly follow the following format to score the reply, with a score range of 0 to 1 points.

[0043] Implication relationship: Indicates whether a conclusion (response) can be necessarily inferred from another premise (question).

[0044] High score: When the conclusion is logically consistent with the premise, the conclusion must be true if the premise is true.

[0045] Low score: The relationship between the two is unclear or cannot be directly inferred.

[0046] Neutral relationship: indicates that the conclusion (response) has nothing to do with the premise (question).

[0047] High score: There is no logical connection between the two, or the premise information is insufficient to draw a conclusion.

[0048] Low score: The conclusion clearly relates to the premises (supports or contradicts them).

[0049] Conflict relationship: indicates that the conclusion (response) is in direct contradiction with the premise (question).

[0050] High score: When the premises are true, the conclusion must be false.

[0051] Low score: There is no obvious conflict between the two.

[0052] Please refer to the scoring rules to score the AI ​​response."

[0053] S302: Input each third text and multiple fourth texts in the multiple second texts, and a preset third prompt word into the large model to obtain a third relevance score between each third text and each fourth text output by the large model.

[0054] The fourth text is a text in the second text that is different from the third text, and the third text is any one of the second texts. The third prompt word can be set or modified by the user at the terminal 101 of the large-scale hallucination detection system 100. The third relevance score includes an implication relationship score, a neutral relationship score, and a conflict relationship score.

[0055] Among them, users can send the preset third prompt word to the selected large model through the API interface, and then obtain the second relevance score output by the large model. For example, different second texts are regarded as different replies, and the prompt words are as follows: "You are an expert in evaluating text quality and will receive multiple replies from AI. Your task is to evaluate the implication relationship score, neutral relationship score and conflict relationship score of different AI replies. You need to first analyze and then strictly follow the following format to score the replies, with a score range of 0 to 1 points.

[0056] Implication relation: Indicates whether a conclusion (response) can be necessarily inferred from another premise (response).

[0057] High score: When the conclusion is logically consistent with the premise, the conclusion must be true if the premise is true.

[0058] Low score: The relationship between the two is unclear or cannot be directly inferred.

[0059] Neutral relationship: indicates that the conclusion (response) has nothing to do with the premise (response).

[0060] High score: There is no logical connection between the two, or the premise information is insufficient to draw a conclusion.

[0061] Low score: The conclusion clearly relates to the premises (supports or contradicts them).

[0062] Conflict relationship: indicates that the conclusion (response) is in direct contradiction with the premise (response).

[0063] High score: When the premises are true, the conclusion must be false.

[0064] Low score: There is no obvious conflict between the two.

[0065] Please refer to the scoring rules to score the AI ​​response."

[0066] S303 : Obtain a second relevance score for each second text according to the third relevance score between each third text and each fourth text.

[0067] Among them, when multiple second texts are input into the large model, the large model will first output the third correlation score of each third text and each fourth text in the multiple second texts, and then calculate the second correlation score of each second text based on the third correlation score of each third text and each fourth text.

[0068] Specifically, the method for calculating the second relevance score of each second text based on the third relevance score between each third text and each fourth text can be to perform weighted averaging or aggregation calculation on the third relevance scores of each third text and all fourth texts to obtain the second relevance score corresponding to the third text, and then average or aggregate the second relevance scores corresponding to all third texts to obtain the final second relevance score of each second text. There is no restriction on the method for calculating the second relevance score of each second text based on the third relevance score between each third text and each fourth text.

[0069] It can be seen that in this example, the first relevance score of the first text and the second text, and the third text and the fourth text within the second text are respectively input into the big model in combination with the preset prompt words, and then the second relevance score of the second text is obtained based on the latter's score. With the help of the semantic understanding ability and multi-level association analysis of the big model, the degree of semantic association between texts can be measured more comprehensively and accurately.

[0070] In one possible implementation, see Figure 4 , Figure 4 This is a flow chart of a method for obtaining a second relevance score for each second text provided by an embodiment of the present application. Figure 4 As shown, obtaining the second relevance score of each second text according to the third relevance score of each third text and each fourth text includes the following steps: S401 : Calculate the average of the third relevance scores of each third text and each fourth text to obtain a second relevance score of each second text.

[0071] The third correlation score between each third text and each fourth text includes a score for indicating the implication relationship between the text and the fourth text, a score for indicating the neutral relationship between the text and the fourth text, and a score for indicating the conflict relationship between the text and the fourth text.

[0072] Here, taking the implication relationship score in the third relevance score as an example, there are five second texts, one of which is selected as the third text. The implication relationship scores of the third text and each fourth text are 0.5, 0.7, 0.4, and 0.6, respectively. By calculating the average of the implication relationship scores within the third text and each fourth text, the implication relationship score in the second relevance score of the third text is obtained. In this case, the implication relationship score is equal to (0.5 + 0.7 + 0.4 + 0.6) / 4 = 0.55. Repeat this process until the second relevance score of each second text is obtained for all second texts.

[0073] In another implementation, in addition to calculating the mean of the third correlation scores of each third text and each fourth text to obtain the second correlation score of each second text, the third correlation scores of each third text and each fourth text in multiple second texts can also be arranged in order from large to small, and then a specific percentile is taken to obtain the second correlation score of each second text. For example, the 75th percentile after arrangement can be directly taken to obtain the second correlation score of each second text.

[0074] It can be seen that in this example, the second correlation score of each second text is obtained by calculating the mean of the third correlation scores of each third text and each fourth text. This can effectively integrate the correlation information between multiple groups of texts and weaken the deviation influence of single data in the form of mean, thereby more comprehensively reflecting the overall correlation degree of the second text.

[0075] S203: Determine, based on the mapping relationship between prompt words and relevance scores, a first prompt word corresponding to each first relevance score and a second prompt word corresponding to each second relevance score.

[0076] Among them, the user can set the mapping relationship between the prompt word and the relevance score at the terminal 101 of the large-scale hallucination detection system 100. Specifically, the mapping relationship between the prompt word and the relevance score can be one-to-one, many-to-one, or one-to-many. When the mapping relationship between the prompt word and the relevance score is many-to-one, if there are multiple prompt words corresponding to a certain relevance score, one of them can be selected as the prompt word. For example, the first relevance score corresponds to prompt word 1 and prompt word 2. In this case, one of them can be selected as the prompt word corresponding to the first relevance score. When the mapping relationship between the prompt word and the relevance score is one-to-many, different relevance score intervals can be set, and the prompt words corresponding to the relevance scores in the same interval are the same.

[0077] For each first relevance score, the corresponding score range or matching rule is searched through the mapping relationship to determine the corresponding first prompt word, which is used to describe the degree of semantic relevance corresponding to the score. Similarly, each second relevance score (e.g., through the mapping rule) is matched to a second prompt word.

[0078] S204: Concatenate the first prompt word corresponding to each first relevance score and the second prompt word corresponding to each second relevance score to obtain a third prompt word corresponding to each second text.

[0079] Among them, when splicing the first prompt word and the second prompt word, the specific splicing order can be set or changed by the user at the terminal 101 of the large model hallucination detection system 100, and the splicing method can be direct combination, or use a separator to connect the two prompt words to obtain the third prompt word corresponding to each second text.

[0080] S205 , performing feature extraction on the third prompt word corresponding to each second text to obtain features of the third prompt word corresponding to each second text.

[0081] In addition to using cue words for feature extraction to obtain the features of the third cue word corresponding to each second text, the cue word text can also be directly converted into a feature vector using word embeddings (such as Word2Vec and GloVe) or sentence embeddings (such as Sentence-BERT). Alternatively, relevance scores can be directly mapped to the dimensions of the feature vector using predefined logical rules. Specifically, the feature vector contains dimensions such as [the implication relationship score between the first and second texts, the neutrality relationship score between the first and second texts, the conflict relationship score between the first and second texts, the implication relationship score between different second texts, the neutrality relationship score between different second texts, and the conflict relationship score between different second texts]. The scores (0-1) are first used directly as feature values ​​or adjusted to a certain range through linear transformations (such as standardization or normalization). The logical relationships between the scores are then calculated. For example, the implication relationship score between the first and second texts minus the conflict relationship score between the first and second texts can reflect logical consistency, while 1 minus the neutrality relationship score between the first and second texts can reflect the relevance between the first and second texts.

[0082] For example, the relevance scores for a second text are: the implication score between the first and second texts = 0.8, the neutrality score between the first and second texts = 0.2, and the conflict score between the first and second texts = 0.1. The feature vector corresponding to the first relevance score for each second text can be constructed as: [0.8, 0.2, 0.1, 0.8-0.1, 1-0.2].

[0083] S206: Input the features of the third prompt word corresponding to each second text into the first model to obtain a first probability of each second text.

[0084] The first probability is used to represent the hallucination detection result of the large model. The user can set a probability threshold and determine the hallucination detection result of the large model based on the relationship between the first probability and the probability threshold.

[0085] The first model can be a Transformer-type model (such as BERT, GPT, T5), a recurrent neural network (RNN / LSTM), a convolutional neural network (CNN), or the like. Preferably, the first model is a BERT-based model. The BERT-based model encodes the first relevance score between the question (i.e., the first text) and the reply (i.e., the second text), the second relevance scores of different replies, and other text features (such as word frequency and syntactic structure) into a vector as input. A multi-layer Transformer encoder captures contextual dependencies in the feature vector, for example, analyzing whether the logical conflict between the reply and the question is positively correlated with the risk of hallucination. Finally, the [CLS] vector is used to output a binary classification result (hallucination / non-hallucination) or a continuous probability value.

[0086] Specifically, BERT is built on the Transformer encoder. Its core architecture consists of an input layer, an encoding layer, and an output layer. The input layer is an embedding layer, which includes token embeddings and position embeddings. Token embeddings use wordpiece segmentation to break the input text into subwords (e.g., "unhappiness" → "un" "happiness"), mapping each subword to a fixed-dimensional vector (e.g., 768 dimensions). Position embeddings add position information to each subword, addressing the Transformer model's inability to model temporal sequences. The output of the input layer is a vector representation resulting from the sum of the token and position embeddings.

[0087] BERT's encoding layer is a multi-layer Transformer encoder (Encoder Layers). BERT is composed of multiple (such as 12 or 24 layers) identical Transformer encoders stacked together. Each layer contains two sub-modules, namely the multi-head self-attention mechanism (Multi-Head Self-Attention) and the feed-forward neural network (Feed-Forward Neural Network).

[0088] The multi-head self-attention mechanism uses multiple attention heads in parallel to capture semantic dependencies between different positions in the text and output context-aware feature vectors. A feedforward neural network performs nonlinear transformations on the self-attention output to further extract features.

[0089] BERT's output layer performs feature aggregation and task adaptation. It adds a special symbol [CLS] to the beginning of the input sequence. The resulting encoded vector (e.g., h_0) serves as the feature representation of the entire sentence for classification tasks such as hallucination probability prediction. The fully connected layer (Dense Layer) in the output layer then linearly transforms the [CLS] vector and applies an activation function (e.g., Sigmoid) to output a hallucination probability value (range: 0-1).

[0090] It can be seen that in this example, the characteristics of the prompt word are generated by combining the correlation between the question and multiple replies output by the model, as well as the correlation between multiple replies, and the first probability is obtained by combining the characteristics of the prompt word and the first model prediction, thereby obtaining the hallucination detection result of the large model, realizing the detection of hallucinations from the perspective of the correlation between replies and the correlation between replies and questions, without relying on reference materials, thereby improving the accuracy of hallucination detection.

[0091] In a possible implementation, after inputting the text features into the target model to obtain the first probability of each second text, the method further includes: If the sum of the first probabilities of each second text is greater than a first value, it is determined that hallucinations exist in the plurality of second texts.

[0092] The first value is half the number of the plurality of second texts. For example, if the number of second texts is 4, the second value is 2. At this time, the sum of the first probabilities of each second text is 2.1. Since 2.1 is greater than 2, it is determined that the plurality of second texts are hallucinations. The first value may also be calculated using other methods and is not limited here.

[0093] Among them, in addition to determining that multiple second texts exist as hallucinations when the sum of the first probabilities of each second text is greater than the first value, when the first probability of a second text exceeds the probability threshold preset by the user, it can be directly determined that multiple second texts exist as hallucinations. For example, if the first probability of a second text exists with a value of 0.9 and the probability threshold preset by the user is 0.85, then 0.9>0.85, and multiple second texts are determined to exist as hallucinations.

[0094] Alternatively, among the multiple second texts output by the first model, when the number of second texts whose hallucination probability is greater than the hallucination probability threshold is greater than a second value, the multiple second texts output by the first model are considered to have hallucinations. The second value can be directly set by the user or obtained by calculation. For example, after the first text is input, the first model outputs 4 second texts, and the hallucination probabilities of different second texts are 0.3, 0.6, 0.2, and 0.8 respectively. The hallucination probability threshold is 0.6. The second value is equal to half of the number of second texts, and the second value is equal to 2. At this time, since the number of second texts whose hallucination probability is greater than the hallucination probability threshold is 1, which is less than the second value, it can be considered that the multiple second texts output by the first model do not have hallucinations.

[0095] It can be seen that in this example, hallucination is determined by calculating the sum of the first probabilities of each second text and comparing it with the first value of half the number of second texts, which can effectively identify the overall hallucination problem of inconsistent content or factual deviation in multiple texts.

[0096] In one possible implementation, the method further includes: If the sum of the hallucination probabilities of each second text is less than or equal to a first value, it is determined that the multiple second texts output by the large model do not contain hallucinations, and a fifth text is determined from the multiple second texts based on the first relevance score and the second relevance score of each second text.

[0097] When the sum of the hallucination probabilities of each second text is less than or equal to half the number of second texts (i.e., the first value), the hallucination probability of most texts is low and the overall content consistency meets the standard. Therefore, the multiple second texts output by the large model are determined to be free of hallucination. At this point, the fifth text that best matches the first text can be found from the multiple second texts output by the large model. If the first text is a question, the fifth text is the best response to the question.

[0098] Specifically, on the basis of confirming that there is no hallucination, according to the first relevance score and the second relevance score of each text, the text with the strongest relevance is screened out from multiple second texts through preset rules (such as weighted summation, taking the maximum value of double scores, etc.) to obtain the fifth text.

[0099] It can be seen that in this example, by using the sum of half the hallucination probabilities as the threshold to determine content consistency and combining individual and overall relevance scores to filter texts, it is possible to efficiently locate the text with the highest relevance to the first text among multiple second texts while ensuring that the large model output has no overall hallucination.

[0100] In one possible implementation, see Figure 5 , Figure 5 This is a flow chart of a method for determining a fifth text from multiple second texts provided by an embodiment of the present application. Figure 5 As shown, determining a fifth text from the plurality of second texts based on the first relevance score and the second relevance score of each second text includes the following steps: S501, calculating and obtaining a weighted sum of the first relevance score and the second relevance score of each second text; S502: Determine the second text with the smallest calculated weighted sum as the fifth text.

[0101] When the plurality of second texts output by the first model do not contain hallucinations, it is necessary to further find the best response that matches the first text from the plurality of second texts.

[0102] For each second text, the weighted sum of its first relevance score and second relevance score is calculated using the formula: Weighted sum = α × first relevance score + β × second relevance score.

[0103] Wherein, α and β are weight coefficients, and α and β can be set or changed by the user at the terminal 101 .

[0104] Among them, when the second text with the smallest calculated weighted sum is determined as the fifth text, if there are multiple second texts whose weighted sums are equal and all are minimum values, the fifth text is further selected based on the first relevance scores of the multiple second texts. For example, the second text with the lowest conflict score with the first text can be directly determined as the fifth text.

[0105] Furthermore, only scores of specific dimensions in the first relevance score and the second relevance score can be selected for weighted sum calculation to determine the fifth text among multiple second texts. For example, the weighted sum of the implication score of each second text with the first text and the implication score of each second text (with other second texts) can be calculated, and the second text with the smallest weighted sum can be determined as the fifth text.

[0106] As can be seen, in this example, after determining that there is no hallucination, the first relevance score of each second text with the first text, as well as the second relevance scores between different second texts, are weighted together to select second texts with reasonable logical deduction and close relevance to the question as responses to the first text. This improves the quality of the output responses while also enhancing processing efficiency through quantitative indicators.

[0107] In one possible implementation, see Figure 6 , Figure 6 This is a flow chart of a method for training a second model to obtain a first model provided by an embodiment of the present application, such as Figure 6 As shown, the method further includes: S601, inputting the feature into a second model to obtain a hallucination detection result of the feature; S602, assigning values ​​to the hallucination detection results to construct training data; S603: Train the second model according to the training data to obtain the first model.

[0108] The second model can be a model with classification or probabilistic prediction capabilities, such as a recurrent neural network model, a convolutional neural network model, and a model based on the Transformer architecture. The specific type of the second model is not limited here. Preferably, the second model is a BERT base model. The network architecture of the BERT base model is introduced in step S206 and will not be repeated here.

[0109] When assigning values ​​to hallucination detection results, a value of 1 can be directly assigned if the hallucination detection result indicates hallucination, and a value of 0 can be assigned if the hallucination detection result indicates no hallucination. The assigned value can then be paired with the text features to obtain training data. Alternatively, when the hallucination detection result is a probability, the value can be determined based on the probability value, such as assigning a value of 0 when the probability value is less than or equal to 0.5 and assigning a value of 1 when the probability value is greater than 0.5.

[0110] The constructed training data is used to iteratively optimize the second model (such as adjusting model parameters through back propagation) to minimize the error between the predicted label and the true label. The final trained model is the first model, which can be used for hallucination recognition in subsequent actual scenarios.

[0111] It can be seen that in this example, by inputting text features into the second model to obtain hallucination detection results and assigning values ​​to construct training data, and then iteratively training to obtain the first model, the supervised learning optimization of the hallucination detection model is achieved, effectively improving the model's ability to recognize text hallucinations.

[0112] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of a large-scale hallucination detection device provided in an embodiment of the present application. Figure 7 As shown, the large model hallucination detection device 700 includes: An acquisition module 701 is configured to input a first text into a macro model to obtain a plurality of second texts that respond to the first text; A processing module 702 is configured to calculate, based on the first text and the second text, a first relevance score between the first text and each of the plurality of second texts and a second relevance score between each of the second texts; and determining, based on a mapping relationship between prompt words and relevance scores, a first prompt word corresponding to each first relevance score and a second prompt word corresponding to each second relevance score; and concatenating the first prompt word corresponding to each of the first relevance scores and the second prompt word corresponding to each of the second relevance scores to obtain a third prompt word corresponding to each of the second texts; and extracting features of the third prompt word corresponding to each second text to obtain text features of each second text; And it is used to input the feature of the third prompt word corresponding to each second text into the first model to obtain the first probability of each second text, and the first probability is used to represent the hallucination detection result of the large model.

[0113] In one possible implementation, in terms of calculating the first relevance score between the first text and each second text in the multiple second texts and the second relevance score of each second text based on the first text and the second text, the processing module 702 is specifically used to: input the first text, each second text in the multiple second texts, and a preset second prompt word into the large model to obtain the first relevance score between the first text output by the large model and each second text in the multiple second texts; input each third text in the multiple second texts, multiple fourth texts, and a preset third prompt word into the large model to obtain the third relevance score between each third text and each fourth text output by the large model, the fourth text being a text in the second text different from the third text, and the third text being any one of the second texts; and obtaining the second relevance score of each second text based on the third relevance score between each third text and each fourth text.

[0114] In one possible implementation, in terms of obtaining the second correlation score of each second text based on the third correlation score of each third text and each fourth text, the processing module 702 is specifically used to: calculate the average of the third correlation scores of each third text and each fourth text to obtain the second correlation score of each second text.

[0115] In one possible implementation, after inputting the features of the third prompt word corresponding to each second text into the first model to obtain the first probability of each second text, the processing module 702 is further used to: if the sum of the first probabilities of each second text is greater than a first value, determine that hallucinations exist in the multiple second texts, wherein the first value is half the number of the multiple second texts.

[0116] In one possible implementation, the processing module 702 is further used to: if the sum of the hallucination probabilities of each second text is less than or equal to a first numerical value, determine that the multiple second texts output by the large model do not have hallucinations, and determine a fifth text from the multiple second texts based on the first relevance score and the second relevance score of each second text.

[0117] In one possible implementation, in determining the fifth text from the multiple second texts based on the first relevance score and the second relevance score of each second text, the processing module 702 is specifically used to: calculate the weighted sum of the first relevance score and the second relevance score of each second text; and determine the second text with the smallest calculated weighted sum as the fifth text.

[0118] In one possible implementation, the processing module 702 is further used to: input the text feature into the second model to obtain the hallucination detection result of the text feature; assign a value to the hallucination detection result to construct training data; and train the second model according to the training data to obtain the first model.

[0119] It is worth noting that the specific functional implementation of the large model hallucination detection device 700 can be found in the above Figure 2 The description of the large-scale model hallucination detection method shown in FIG. For example, the acquisition module 701 is used to implement the relevant content of executing S201, and the processing module 702 is used to implement the relevant content of executing S202-S206. The various units or modules in the large-scale model hallucination detection device 700 can be individually or completely combined into one or more other units or modules to form a structure, or one or more of the units or modules can be further divided into multiple functionally smaller units or modules to form a structure, which can achieve the same operation without affecting the technical effects of the embodiments of the present invention. The above-mentioned units or modules are divided according to logical functions. In actual applications, the functions of one unit (or module) are implemented by multiple units (or modules), or the functions of multiple units (or modules) are implemented by one unit (or module).

[0120] According to the description of the above method embodiment and related device embodiment, please refer to Figure 8 , Figure 8 It is a structural diagram of a computer provided in an embodiment of the present application. Figure 8 The computer 800 shown includes a processor 801 , a memory 802 , a communication interface 803 , and a bus 804 . The processor 801 , the memory 802 , and the communication interface 803 are communicatively connected to each other via the bus 804 .

[0121] Optionally, the memory 802 is a ROM, a static storage device, a dynamic storage device or a RAM.

[0122] The memory 802 can store executable program codes. When the executable program codes stored in the memory 802 are executed by the processor 801, the processor 801 and the communication interface 803 are used to execute the program codes. Figure 2 The various steps of the large model hallucination detection method of the illustrated embodiment.

[0123] The processor 801 adopts a general CPU, a microprocessor, an application-specific integrated circuit ASIC, a GPU or one or more integrated circuits to execute relevant programs to perform the large model hallucination detection method of the method embodiment of the present application.

[0124] Processor 801 can also be an integrated circuit chip with signal processing capabilities. During implementation, each step of the large-scale hallucination detection method of the present application can be completed by hardware integrated logic circuits or software instructions in processor 801. Optionally, processor 801 is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The processor can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor is a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The optional software module is located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other storage media well-known in the art. The storage medium is located in the memory 802, and the processor 801 reads the information in the memory 802, and combines its hardware to complete the functions required to be performed by the modules included in the large model hallucination detection device 700 of an embodiment of the present application, or executes the large model hallucination detection method of the method embodiment of the present application.

[0125] The communication interface 803 uses, for example but not limited to, a transceiver or other transceiver-related device.

[0126] The bus 804 may include a path for transmitting information between various components of the computer 800 (eg, the memory 802 , the processor 801 , and the communication interface 803 ).

[0127] It should be noted that although Figure 8The computer 800 shown only shows a memory, a processor, and a communication interface. However, in the specific implementation process, those skilled in the art should understand that the computer 800 also includes other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the computer 800 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the computer 800 may also include only the devices necessary to implement the embodiments of the present application, and does not necessarily include Figure 8 All devices shown in .

[0128] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program for electronic data exchange. The computer program includes execution instructions, and the execution instructions are used to execute part or all of the steps of any large-model hallucination detection method described in the above-mentioned large-model hallucination detection method embodiment. The above-mentioned computer includes an electronic terminal device.

[0129] An embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and the computer program is operable to enable a computer to perform part or all of the steps of any large model hallucination detection method recorded in the above method embodiments. The computer program product can be a software installation package.

[0130] It should be noted that for the aforementioned embodiments of the hallucination detection method for any large model, for the sake of simplicity, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that this application is not limited to the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0131] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of a large-scale hallucination detection method, computer, storage medium, and program product of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, based on the ideas of a large-scale hallucination detection method, computer, storage medium, and program product of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present application.

[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, hardware products, and computer program products of the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The memory may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0134] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality of components or steps. The fact that certain measures are recited in different dependent claims does not mean that these measures cannot be combined to produce good results.

[0135] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above-mentioned large-model hallucination detection method embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0136] It can be understood that any product that is controlled or configured to execute the processing method of the flowchart described in an embodiment of a large-model hallucination detection method of the present application, such as the device and computer program product of the above flowchart, falls within the scope of the related products described in the present application.

[0137] Obviously, those skilled in the art may make various modifications and variations to the large-scale hallucination detection method, computer, storage medium, and program product provided herein without departing from the spirit and scope of the present application. Thus, if such modifications and variations fall within the scope of the claims and their equivalents, the present application is intended to encompass such modifications and variations.

Claims

1. A large model hallucination detection method, characterized in that: The method comprises: Inputting a first text into a large model to obtain a plurality of second texts that respond to the first text; Calculating, based on the first text and the second text, a first relevance score between the first text and each of the plurality of second texts and a second relevance score between each of the second texts; Determining, based on a mapping relationship between prompt words and relevance scores, a first prompt word corresponding to each first relevance score and a second prompt word corresponding to each second relevance score; concatenating the first prompt word corresponding to each of the first relevance scores and the second prompt word corresponding to each of the second relevance scores to obtain a third prompt word corresponding to each of the second texts; Performing feature extraction on the third prompt word corresponding to each second text to obtain features of the third prompt word corresponding to each second text; The features of the third prompt word corresponding to each second text are input into the first model to obtain a first probability of each second text, where the first probability is used to represent the hallucination detection result of the large model.

2. The method according to claim 1, wherein The step of calculating, based on the first text and the second text, a first relevance score between the first text and each of the plurality of second texts and a second relevance score between each of the second texts includes: Inputting the first text, each second text in the plurality of second texts, and a preset second prompt word into the large model, and obtaining a first relevance score between the first text and each second text in the plurality of second texts output by the large model; Inputting each third text and a plurality of fourth texts in the plurality of second texts, and a preset third prompt word into the large model, and obtaining a third relevance score between each third text and each fourth text output by the large model, wherein the fourth text is a text in the second text that is different from the third text, and the third text is any one of the second texts; A second relevance score for each second text is obtained according to the third relevance score between each third text and each fourth text.

3. The method according to claim 2, wherein Obtaining a second relevance score for each second text according to the third relevance score between each third text and each fourth text includes: An average of the third relevance scores of each third text and each fourth text is calculated to obtain a second relevance score of each second text.

4. The method according to claim 1, wherein After inputting the feature of the third prompt word corresponding to each second text into the first model to obtain the first probability of each second text, the method further includes: If the sum of the first probabilities of each second text is greater than a first value, it is determined that hallucinations exist in the plurality of second texts, wherein the first value is half of the number of the plurality of second texts.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: If the sum of the hallucination probabilities of each second text is less than or equal to a first value, it is determined that the multiple second texts output by the large model do not contain hallucinations, and a fifth text is determined from the multiple second texts based on the first relevance score and the second relevance score of each second text.

6. The method according to claim 5, wherein The determining a fifth text from the plurality of second texts based on the first relevance score and the second relevance score of each second text includes: Calculating a weighted sum of the first relevance score and the second relevance score of each second text; The second text with the smallest calculated weighted sum is determined as the fifth text.

7. The method according to any one of claims 1 to 6, wherein: The method further comprises: inputting the feature into a second model to obtain a hallucination detection result of the feature; Assigning values ​​to the hallucination detection results to construct training data; The second model is trained according to the training data to obtain the first model.

8. A computer, characterized in that: include: A memory, a processor, and an executable program code stored in the memory and capable of running on the processor, wherein the processor executes the steps of the large model hallucination detection method according to any one of claims 1 to 7 when executing the executable program code.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores executable program code, which includes execution instructions for executing the steps of the large model hallucination detection method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises a computer program, and the computer program is used to enable a computer to execute the steps of the large model hallucination detection method according to any one of claims 1 to 7.

Images