Model testing methods and related equipment

Abstract

The present application discloses a model detection method and related equipment to solve the problem of how to identify whether a large model has AI hallucination in the prior art. The method includes: obtaining a first prediction statement output by the model, and determining a plurality of proper nouns included in the prediction statement; determining a first compatibility degree of each of the proper nouns to obtain a plurality of first compatibilities, the first compatibility degree being used to characterize the degree of compatibility between the semantic element of the proper noun and the semantic element of the first word, the first word and the proper noun belonging to the first prediction statement; determining whether the model has AI hallucination based on the plurality of first compatibilities and the numerical distribution presented by the plurality of first compatibilities.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model detection method, apparatus, device, computer-readable storage medium, and computer program product. Background Art

[0002] A large model refers to a deep neural network model with millions or billions of parameters. This model has undergone a special training process and is capable of performing complex processing and task handling on large-scale data.

[0003] In actual scenarios, when asking a big model about a non-existent historical event, the big model may also provide a fictitious prediction statement. This phenomenon is called artificial intelligence (AI) hallucination.

[0004] How to identify whether a large model has AI hallucinations so as to perform corresponding operations based on the recognition results, such as correcting the prediction sentences output when AI hallucinations occur and retraining the large model, is a problem that needs to be solved urgently. Summary of the invention

[0005] Embodiments of the present application provide a model detection method, apparatus, device, computer-readable storage medium, and computer program product.

[0006] The present application embodiment adopts the following technical solutions: A model detection method comprises: obtaining a first prediction statement output by a model, and determining a plurality of proper nouns included in the first prediction statement; determining a first compatibility degree of each of the proper nouns to obtain a plurality of first compatibilities, wherein the first compatibility degree is used to characterize the degree of compatibility between a semantic element of the proper noun and a semantic element of a first word, wherein the first word and the proper noun belong to the first prediction statement; and determining whether the model has an artificial intelligence hallucination based on the plurality of first compatibilities and the numerical distribution of the plurality of first compatibilities.

[0007] A model detection device comprises: an acquisition unit, used to acquire a first prediction statement output by a model, and to determine a plurality of proper nouns included in the first prediction statement; a compatibility determination unit, used to determine a first compatibility of each of the proper nouns, and obtain a plurality of first compatibilities, wherein the first compatibility is used to characterize the degree of compatibility between a semantic element of the proper noun and a semantic element of a first word, wherein the first word and the proper noun belong to the first prediction statement; and a detection result determination unit, used to determine whether the model has an artificial intelligence hallucination based on the plurality of first compatibilities and the numerical distribution presented by the plurality of first compatibilities.

[0008] A computing device comprises: a memory and a processor, wherein the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program stored in the memory to perform the above method.

[0009] A computer-readable storage medium storing a computer program, wherein the computer program can implement the above method when executed by a computer.

[0010] A computer program product stores instructions, which, when executed by a computer, enable the computer to implement the above method.

[0011] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: If the AI ​​model does not have AI hallucination, the first compatibility of the proper nouns in the predicted sentence it outputs will be higher; at the same time, the numerical distribution presented by the first compatibilities of multiple proper nouns will also match the expected numerical distribution type. Based on the above characteristics, the embodiment of the present application uses "the first compatibility of multiple proper nouns" and "the numerical distribution presented by multiple first compatibilities" as the basis for judging whether the AI ​​model has AI hallucination, thereby providing a solution for effectively identifying whether a large model has AI hallucination. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1a A specific implementation flow chart of a model detection method provided in an embodiment of the present application; Figure 1b A schematic diagram of a possible distribution of values ​​of a plurality of first compatibilities in an embodiment of the present application; Figure 1c A schematic diagram of another possible situation of numerical distribution of multiple first compatibilities in an embodiment of the present application; Figure 2 A schematic diagram of a scenario architecture of an actual scenario in which the model detection method provided in the embodiment of the present application can be used; Figure 3 A schematic diagram of a specific application process of the model detection method provided in an embodiment of the present application in an actual scenario; Figure 4 A schematic diagram of the specific structure of a model detection device provided in an embodiment of the present application; Figure 5 A schematic diagram of the specific structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0014] Those skilled in the art may appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0015] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0016] The embodiment of the present application provides a model detection method to solve the problem of how to identify whether a large model has AI hallucinations in the prior art.

[0017] The executor of the method may be any computing device that can implement the method, such as a server, a mobile phone, a personal computer, a smart wearable device, a smart robot, and the like.

[0018] The different steps of the method may be implemented by the same execution subject or by different execution subjects. The embodiment of the present application does not limit the execution subject used to implement the method.

[0019] In addition, the execution order of different steps is not limited in the embodiment of the present application. When using the method provided in the embodiment of the present application, the execution order of different steps can be adjusted according to actual needs.

[0020] For ease of description, the method provided by the embodiment of the present application is described in detail below by taking a client that can call a model as an execution subject of the method as an example. The model, for example, can be a large model.

[0021] like Figure 1aAs shown, a specific implementation flow chart of a model detection method provided in an embodiment of the present application includes the following steps: Step 11: obtaining a first prediction sentence output by the model, and determining a plurality of proper nouns included in the first prediction sentence; In an optional implementation, the model described in the embodiments of the present application may be any model that performs natural language processing (NLP) tasks. The NLP tasks may include, but are not limited to: generating text summaries, answering user questions, continuing documents, etc.

[0022] In an optional implementation, the model may include but is not limited to: Large Language Model (LLM). LLM may include, but is not limited to: BERT (Bidirectional Encoder Representation from Transformers, a pre-trained language model based on a multi-layer Transformer encoder) model, GPT model, T5 model, PaLM2, Llama2, and the like.

[0023] In the embodiment of the present application, the model may have the ability to generate natural language text, and the generated natural language text may be referred to as a prediction sentence. For the convenience of distinguishing descriptions, the prediction sentence described in step 11 is referred to as a first prediction sentence. After the first prediction sentence is generated, it may be output by the model.

[0024] In some specific examples, the model can generate a corresponding answer to a question input by a user, and the answer to the question is the first predicted sentence; or, based on a text input by a user, a summary of the text is generated, and the summary is the first predicted sentence; or, a text input by a user can be continued, and the continued content is the first predicted sentence, etc. Generally, the first predicted sentence can refer to one sentence, or the first predicted sentence can also refer to several sentences.

[0025] In the embodiment of the present application, the first prediction sentence may include one or more proper nouns. The proper nouns included in the first prediction sentence generally refer to nouns with specific meanings or referring to specific things, as opposed to common nouns.

[0026] In an optional implementation, a proper noun is usually used to refer to something or a concept that is unique in the world, such as a specific person's name, place, organization, group, country, event, time (such as a festival) or a specific object, etc.

[0027] In an optional implementation, the embodiment of the present application can pre-set a determination condition for "proper nouns", and determine whether the word in the first predicted sentence is a proper noun based on the condition. Specifically, a word that meets the determination condition can be determined as a proper noun; a word that does not meet the determination condition can be determined as a common noun.

[0028] In a specific example, "having a specified part of speech and specified semantics" may be set as a determination condition for proper nouns.

[0029] In another optional implementation, a proper noun library may be provided to implement the determination of proper nouns. The proper noun library stores words pre-selected as proper nouns. By segmenting the first predicted sentence and then comparing it with the words in the proper noun library, the words obtained by segmentation that are consistent with the words in the proper noun library can be determined as proper nouns.

[0030] In another optional implementation, the first predicted sentence may also be input into a proper noun detection model, and the model may be used to recognize / determine the proper noun in the first predicted sentence.

[0031] In the embodiment of the present application, the proper noun detection model may include two sub-models, namely: a named entity model and a special discrimination model. The functions of the two sub-models are respectively introduced below.

[0032] 1. Named entity model, used to identify the type of named entities and event types contained in the prediction sentence of the input named entity model, and output: named entity type, event type, index of named entity, index of event type.

[0033] Named entities refer to names of people, organizations, places, and all other entities identified by names. More general entities also include numbers, dates, currencies, addresses, and so on.

[0034] If the named entity type and the corresponding named entity are expressed as "type-entity", some specific examples are: time-February 14, place name-Shanghai, holiday-Valentine's Day, person name-Li Si, organizational structure-Chinese Academy of Sciences, number-100%, and so on.

[0035] An event is a form of information, which is defined as the fact that specific people and objects interact with each other at a specific time and place. For example, if a text contains the words "fire" and "a certain country's tank", then "fire" is the trigger word of the event, "a certain country's tank" is the argument, and the argument role of the argument is "tool". Based on this information, the type of event contained in the text can be identified as "attack".

[0036] The named entity model can identify the type of named entities contained in the prediction sentence by performing the named entity recognition (NER) task; at the same time, it can identify the type of events contained in the first prediction sentence by performing the event extraction task.

[0037] In an optional implementation, the named entity model may be a pre-trained model, such as a RoBERTa model.

[0038] The index of the named entity is used to indicate the position of the named entity in the first prediction sentence; the index of the event type is used to indicate the position of the sentence describing "the event corresponding to the event type" in the first prediction sentence.

[0039] 2. A special discriminant model is used to take the named entity type, event type, index of the named entity, and index of the event type output by the named entity model as input, and output the correlation value of the named entity type and the event type.

[0040] From the implementation principle of the special discrimination model, it is equivalent to inferring whether which named entities are special based on the combination of named entity types and event types, combining the index of named entities and the index of event types, that is, inferring the named entities generated by the model when AI hallucination occurs.

[0041] Generally, the larger the correlation value output by the special discriminant model, the greater the probability that the named entity is generated by the model when AI hallucination occurs; the smaller the correlation value, the smaller the probability. In the case of presetting a correlation value threshold, the named entity corresponding to the correlation value greater than the correlation value threshold can be determined as the proper noun included in the first prediction sentence.

[0042] In an optional implementation, the special discriminant model can be trained using the following steps: Step ①: Using the pre-trained language model, obtain the named entities, indexes of named entities, event types, and indexes of event types in the prediction sentences output by the generative artificial intelligence model without AI hallucination; Step ②: annotate the first correlation value for the combination of the named entity, the index of the named entity, the event type, and the index of the event type obtained by executing step ① as a positive sample; Step 3: Using the pre-trained language model, obtain the named entities, indexes of named entities, event types, and indexes of event types in the prediction sentences output by the generative artificial intelligence model when AI hallucination occurs; Step ④: annotate the combination of the named entity, the index of the named entity, the event type, and the index of the event type obtained by executing step ③ with a second correlation value as a negative sample; wherein the second correlation value is less than the first correlation value; Step ⑤: Use positive and negative samples as inputs of the special discriminant model to be trained, and use the correlation value between the named entity and the event type in the combination as the prediction target to obtain a trained special discriminant model.

[0043] Step 12: Determine the first compatibility of each proper noun to obtain a plurality of first compatibilities; The first compatibility of a proper noun, which may also be referred to as the semantic compatibility of a proper noun, is used to characterize the degree of compatibility between the semantic element of the proper noun and the semantic element of the first word. The first word and the proper noun belong to the first predicted sentence. In other words, the first compatibility of a proper noun is used to characterize the degree of compatibility between the semantic element of the proper noun and the semantic element of "other words that belong to the first predicted sentence together with the proper noun".

[0044] When the predicted sentence contains multiple proper nouns, multiple corresponding first compatibilities can be obtained for the multiple proper nouns, such as: the first compatibility a of proper noun A, the first compatibility b of proper noun B, the first compatibility c of proper noun C, ... and so on.

[0045] Generally speaking, the greater the first compatibility of a proper noun, the higher the compatibility between the proper noun and other words in the first prediction sentence, the higher the rationality of the proper noun and other words in the first prediction sentence appearing in the first prediction sentence, and the lower the probability of AI hallucination occurring in the LLM outputting the first prediction sentence; conversely, it indicates that the proper noun appears more abruptly in the first prediction sentence, and the probability of AI hallucination occurring in the LLM is higher.

[0046] In an optional implementation, the first compatibility of the proper noun can be determined based on the second compatibility (also called semantic compatibility) and the third compatibility (also called part-of-speech compatibility) of the proper noun. The second compatibility represents the degree of compatibility between the semantics of the semantic element of the proper noun and the semantic element of the first word; the third compatibility is used to represent the degree of compatibility between the part-of-speech of the proper noun and the part-of-speech of the first word.

[0047] In a specific example, the first compatibility=semantic compatibility+part of speech compatibility; or, the first compatibility= α Semantic compatibility+ β Part-of-speech compatibility, where α and βFor example, in the embodiment of the present application, for the first predicted sentence S, the first compatibility of the proper noun X therein can be obtained by adding the compatibility of the semantic and grammatical dimensions, that is, the first compatibility of X = the semantic compatibility of X in S + the grammatical compatibility of X in S.

[0048] In the embodiments of the present application, the semantic compatibility and the grammatical compatibility can be respectively calculated based on the compatibility means of the three dimensions of the corresponding semantics / grammar by taking the weighted sum of the compatibility means of the three dimensions.

[0049] Specifically, the mean values ​​of the compatibility of the three semantic dimensions refer to: the mean value of the semantic compatibility between X and all other words in S, the mean value of the semantic compatibility between X and the important components in S, and the mean value of the semantic compatibility between X and the n words before and after S.

[0050] The mean compatibility of the three dimensions of grammar refers to: the mean grammatical compatibility between X and all other words in S, the mean grammatical compatibility between X and the important components in S, and the mean grammatical compatibility between X and the n words before and after S.

[0051] Semantic compatibility can be used to indicate whether the semantic elements of different words constituting the same sentence are compatible and the degree of compatibility. Optionally, the greater the semantic compatibility of a sentence (such as the first predicted sentence), the higher the degree of compatibility between the semantic elements of the words constituting the sentence. Optionally, the greater the semantic compatibility of a word in a sentence, the higher the degree of compatibility between the semantic elements of the word and other words in the sentence.

[0052] Semantic compatibility can be calculated based on the semantic element vector of the word. For example, taking the sentence (such as the first predicted sentence) "I am a cat" as an example, the sentence is composed of word A "I", word B "is" and word C "cat", wherein it is assumed that the semantic element vector of word A is {[semantic element 1][semantic element 2]}, the semantic element vector of word B is {[semantic element 3][semantic element 4][semantic element 5]}, and the semantic element vector of word C is {[semantic element 6][semantic element 7]}, then, for word A, the semantic element vector of word A {[semantic element 1][semantic element 2]} and the semantic element vector of word B {[semantic element 3][semantic element 4][semantic element 5]} can be dot-multiplied (that is, the inner product / scalar product of the vectors) to obtain a scalar as the first semantic compatibility P, and the semantic element vector of word A {[semantic element 1][semantic element 2]} and the semantic element vector of word C {[semantic element 6][semantic element 7]} are dot-multiplied to obtain the second semantic compatibility Q. Compute the mean of P and Q as the mean of the grammatical compatibility of A with all other words in the sentence “I am a cat”.

[0053] Similarly, the mean value of the semantic compatibility between a word and important components in a sentence and the mean value of the semantic compatibility between a word and the n words before and after it in a sentence can also be calculated in a similar manner as described above.

[0054] Grammatical compatibility can be used to indicate whether the parts of speech of different words constituting the same sentence are compatible and the degree of compatibility. Optionally, the greater the grammatical compatibility of a sentence, the higher the degree of compatibility between the parts of speech of the words constituting the sentence. Optionally, the greater the grammatical compatibility of a word in a sentence, the higher the degree of compatibility between the parts of speech of the word and other words in the sentence.

[0055] Part of speech refers to the category or type to which a word belongs grammatically, and can represent the grammatical features of a word. In some embodiments, the part of speech of each word in a sentence can be determined by part-of-speech tagging (Part-of-Speech Tagging or POS tagging). Part-of-speech tagging, also known as word class tagging or simply tagging, refers to the method of marking a correct part of speech for each word in a sentence, that is, the process of determining whether each word is a noun, verb, adjective or other part of speech. For example, for the sentence "I am a cat" in the above example, after part-of-speech tagging, it can be determined that the part of speech of word A "I" is a noun, the part of speech of word B "is" is a verb, and the part of speech of word C "cat" is a noun. In the embodiment of the present application, the grammatical compatibility of each proper noun in the predicted sentence can be determined according to the part of speech of the proper noun. In an optional implementation, the following method may be used to determine the grammatical compatibility of each proper noun in the first test sentence S according to the part of speech of the proper noun: First, a word combination consisting of a proper noun X and other words is first input into a pre-trained model with the function of "obtaining a part-of-speech feature vector of a word combination" to obtain a part-of-speech feature vector combination corresponding to the word combination output by the pre-trained model; wherein the part-of-speech feature vector combination is composed of the part-of-speech feature vectors of the proper noun X and the other words; Then, the part-of-speech feature vector is input into a pre-trained neural network with the function of "inferring the acceptability of word combinations", and the neural network infers the acceptability of the word combination as the grammatical compatibility between X and the other word; By analogy, we can obtain: the mean grammatical compatibility between X and all other words in S, the mean grammatical compatibility between X and the important components in S, and the mean grammatical compatibility between X and the n words before and after S; Finally, by calculating the weighted sum of the means of these three grammatical compatibilities, the grammatical compatibility of X in the predicted sentence S can be obtained.

[0056] In an optional implementation, the neural network for the above-mentioned "reasoning word combination acceptability" function may be called an acceptability model, which may be trained using the following steps: Step 1: Based on linguistic experience, sort out the word combinations formed by the words of known parts of speech in a "combination" manner; When words of known parts of speech are combined in a "combination" manner, different word combinations will be generated if the words are arranged in different orders. For example, the two words "enjoy" and "sunshine" can form word combination 1 "enjoy the sunshine" and word combination 2 "sunshine enjoyment".

[0057] Step ②: Mark the acceptability of each word combination; For example, the acceptability of the word combination can be marked with a score in the range of [1, 5], wherein the higher the marked score is, the higher the acceptability of the word combination is.

[0058] A specific example is: word combination 1 "enjoy the sunshine" can be marked with a score of 5; word combination 2 "enjoy the sunshine" can be marked with a score of 1.

[0059] Step 3: By encoding words of different parts of speech, the word combination is converted into a part-of-speech feature vector combination as a training sample. At the same time, the annotated scores of the word combination are used as labels to train the acceptability model to obtain a trained acceptability model.

[0060] After the calculation of the semantic compatibility and the grammatical compatibility of the proper noun in the first predicted sentence is completed, the first compatibility of the proper noun can be calculated based on the semantic compatibility and the grammatical compatibility.

[0061] What needs to be explained about "semetic element" is that the current NLP research and model recognition exploration are stagnant at the word level, and the semantic units below the word are shrouded in mystery. However, from the actual language situation, the word, as the smallest unit that can be used independently, has a subordinate concept at the semantic level, namely, semantic element (SEME).

[0062] Sememe is a term in modern semantics, which refers to the meaning unit (content unit) in a language. It is also called sememe and is the constituent factor of a word.

[0063] When comparing the same word with different words, different numbers of semantic elements can be used to represent it according to usage requirements. For example, if the word "woman" is limited to three semantic elements, it can be represented as [person] [female] [adult]. It can be seen that by analyzing semantic elements, the semantics of the text can be studied at a smaller granularity, so that the true meaning of each word in the text can be more accurately identified.

[0064] Semantic features can be used to distinguish multiple meanings of the same word, while semantic elements can be used to decompose the same meaning of the same word, enabling the study of text semantics at a smaller granularity. Among them, a word can correspond to one or more semantic elements. In addition, polysemous words may have different meanings in different sentences. Therefore, if there are polysemous words in a sentence, the semantic elements of the polysemous word can be determined based on the semantic elements of the words before and after the polysemous word. In some implementations, a semantic element vector may be used to represent a word. The semantic element vector of a word is used to represent the semantics of the word, specifically, to represent the word in a manner similar to a word vector, and is a subordinate representation of the word vector representation.

[0065] Exemplarily, the first predicted sentence "I am a cat" consists of word A "I", word B "is" and word C "cat". Assuming that word A is represented by semantic elements as [semantic element 1] [semantic element 2], word B is represented by semantic elements as [semantic element 3] [semantic element 4] [semantic element 5], and word C is represented by semantic elements as [semantic element 6] [semantic element 7], word A can be represented by a 2-dimensional semantic element vector {[semantic element 1], [semantic element 2]}, word B can be represented by a 3-dimensional semantic element vector {[semantic element 3] [semantic element 4] [semantic element 5]}, and word C can be represented by a 2-dimensional semantic element vector {[semantic element 6] [semantic element 7]}.

[0066] In an optional implementation, the semantic element vector of each word can be determined by querying a pre-built semantic element vector library based on the word. The semantic element vector library is a database for correspondingly storing words and semantic element vectors of words, and the specific form of the database can be a data table.

[0067] In an optional implementation, the semantic vector library may be constructed as follows: First, obtain the annotated corpus.

[0068] The annotated corpus includes a plurality of sentence pairs, each of which includes: a correct sentence, a sentence whose acceptability is greater than a first acceptance threshold after one or more first target words in the correct sentence are replaced by a first specific word, a sentence whose acceptability is greater than a second acceptance threshold after one or more second target words in the correct sentence are replaced by a second specific word, and a sentence whose acceptability is less than a second acceptance threshold after one or more third target words in the correct sentence are replaced by a third specific word. The second acceptance threshold is less than the first acceptance threshold.

[0069] The annotation information of each sentence pair in the annotated corpus includes: the overall acceptability of the sentence (0-5), the number of words incompatible with other words in the sentence & the compatibility of other words incompatible with the word (1-5), and whether the part of speech of the replacement word is the same as that of the replaced word (or the part of speech distance 0-5).

[0070] The overall acceptability of a sentence refers to the degree of coherence of the sentence in terms of grammar and semantics as determined by native speakers. In an optional implementation, the higher the overall acceptability of a sentence, the higher the degree of coherence of the sentence in terms of grammar and semantics; conversely, the lower the degree of coherence of the sentence in terms of grammar and semantics.

[0071] Incompatibility between words refers to the situation where there is a semantic conflict between words. In an optional implementation, the semantic element vectors of two words can be obtained separately. If the similarity between the semantic element vectors of the two words is less than a preset semantic element vector similarity threshold (such as -0.5), then the two words can be determined to be incompatible; if it is greater than the threshold, then the two words can be determined to be compatible. Alternatively, the word vectors of the two words can be obtained separately. If the similarity between the word vectors of the two words is less than a preset word vector similarity threshold, then the two words can be determined to be incompatible; if it is greater than the threshold, then the two words can be determined to be compatible.

[0072] After obtaining the annotated corpus, the semantic element vector of the word can be determined for the word in the annotated corpus, and the word and the semantic element vector of the word can be stored in the semantic element vector database accordingly. In an optional implementation, the specific implementation steps are as follows: Step (1): Take the word "apple" in a sentence pair in the annotated corpus as an example. First, determine the semantic elements of the word "apple" (called original semantic elements), such as [semantic element 1] [semantic element 2] [semantic element 3]; Here, it is assumed that [semantic element 1] is a semantic element representing a mobile phone brand.

[0073] The vector constructed based on the original semantic element is called the original semantic element vector. For example, the original semantic element vector of the word "apple" is {[semantic element 1][semantic element 2][semantic element 3]}.

[0074] Step (2): Based on the original semantics, add (or subtract) semantics. For example, assume that [semetic 0] representing "fruit" is added, so that the semantics of the word "apple" include: [semetic 0] [semetic 1] [semetic 2] [semetic 3]. At this time, the semantic vector of the word "apple" can be expressed as {[semetic 0] [semetic 1] [semetic 2] [semetic 3]}; It should be noted that the word "apple" may refer to a fruit called apple, or it may refer to a mobile phone called apple, that is to say, the current semantic element vector {

semantic element 0

semantic element 1

semantic element 2

semantic element 3

semantic element 0

semantic element 1

semantic element 2

semantic element 3

[0075] Step (3): From the sentence “I like to eat apples”, determine the word “eat” that has a combination relationship with the word “apple”; The explanation of the combination relationship is as follows: Combination relationship: For example, "watch" and "TV" can appear in the same sentence, for example: I like watching TV. Moreover, the overall acceptability of the sentence in which both appear at the same time is high, for example, greater than the preset acceptability threshold. Therefore, there is an aggregation relationship between "watch" and "TV".

[0076] Similarly, there is a combination relationship between "eat" and "apple".

[0077] Step (4): According to the semantic element 0 representing "fruit", obtain the word "pear" containing semantic element 0 from the vocabulary; replace the word "apple" in the sentence with the word "pear", thereby obtaining the sentence "I like to eat pears"; The word library is used to store words with the same or similar semantics and / or words with the same or similar parts of speech.

[0078] Based on actual needs, the vocabulary database can be built according to the official dictionary of a certain language (such as the Modern Chinese Dictionary).

[0079] In the embodiment of the present application, the vocabulary built can be of different types. In a specific example, there can be a vocabulary with the same / similar semantics - every two words in the vocabulary satisfy the "graph relationship between words by adding or / and reducing a certain semantic element to become another word"; there can also be a vocabulary with the same / similar part of speech - every two words in the vocabulary satisfy the "graph relationship between words by adding or / and reducing a certain grammatical semantic element to become another word".

[0080] Step (5): For the sentence “I like to eat pears”, if its overall acceptability is judged to be high, for example, greater than a preset acceptability threshold, it means that there is an aggregation relationship between the word “apple” and the word “pear”; The explanation of the aggregation relationship is as follows: Aggregation relationship: For example, "apple" and "pear" can appear in the same context, for example: I like to eat. The word to be filled in the blank can be apple or pear, and no matter whether the word is apple or pear, the overall acceptability of the sentence is high, for example, greater than the preset acceptability threshold, so there is an aggregation relationship between apple and pear.

[0081] Step (6): Based on the semantic element [semane 0] representing “fruit” that is determined to be shared by the word “pear” that has an aggregation relationship with the word “apple”, the semantic element [semane 1] representing the brand of mobile phone is deleted from the current semantic element vector {[semane 0] [semane 1] [semane 2] [semane 3]} of the word “apple”, thereby obtaining the semantic element vector {[semane 0] [semane 2] [semane 3]} suitable for the context of the current sentence “I like to eat apples”; Step (7): Store the word "apple" and the semantic element vector {[semantic element 0] [semantic element 2] [semantic element 3]} into a semantic element vector library corresponding to the acquired annotated corpus.

[0082] Similarly, the semantic element vector of each word in the annotated corpus can be determined and stored in the semantic element vector library, thus completing the construction of the semantic element vector library.

[0083] It should be noted that, taking word A as an example, the original semantic element vector of word A can be obtained by the following steps: Step a: Use the Transformer structure with a multi-head mechanism to perform positional embedding to obtain the positional encoding information of word A. Then, perform a matrix operation on the word vector of word A and the positional encoding information of word A to obtain the word vector with position information; Step b: Input the word vector with position information of word A into the encoding layer of Transformer. After performing attention calculation at each layer in the multi-head mechanism, a new word vector of the word can be obtained. It should be noted that after attention, the word can obtain semantic association information with other words, and this information is entered into the new word vector of the word.

[0084] Similarly, after all the words in the vocabulary are calculated through attention at each layer in the multi-head mechanism, the corresponding new word vectors can be obtained. The new word vectors will contain semantic association information with other words.

[0085] Step c: Match the new word vector of word A with the new word vectors of all words in the word library in terms of vector segments (or local vectors) (for example, match the vector segments of the 200th to 230th dimensions in the word vector). If the matching result shows that the similarity of the vector segments is within the set similarity range, it is determined that the match is successful, and the successfully matched vector segment is used as a semantic element of word A (or a vector used to represent the semantic element). According to the above matching method, by executing step c, each semantic element of word A can be obtained. Each semantic element can be represented by a vector (that is, a successfully matched vector segment).

[0086] Step d: Arrange all the semantic elements of word A horizontally from left to right to obtain the semantic element vector of word A (original semantic element vector).

[0087] All semantic elements are arranged horizontally from left to right, and the semantic element vector of word A is obtained, that is, a vector sequence obtained by arranging the vectors representing the semantic elements horizontally from left to right.

[0088] In an optional implementation, the semantic vector of a sentence may be acquired and output by adding a semantic vector layer and a semantic vector acquisition interface to the model.

[0089] In an optional implementation, the semantic element vector model can perform word / character embedding processing on each proper noun in a Word2vect manner to obtain a word vector for each proper noun. Then, the semantic element vector for each proper noun can be obtained from a pre-built semantic element vector library.

[0090] After determining the semantic element vector of the word, the first compatibility may be further determined in the manner described above.

[0091] Step 13: Based on the multiple first compatibilities and the numerical distributions presented by the multiple first compatibilities, determine whether the model has artificial intelligence hallucinations.

[0092] In an optional implementation, the numerical distribution mentioned here may refer to a probability distribution. It is understandable that, for the first prediction statement, which often contains multiple proper nouns, after determining the first compatibility of multiple proper nouns (such as all proper nouns in all first prediction statements) in the first prediction statement, the distribution characteristics of the overall probability distribution formed by these first compatibilities on the number of multiple proper nouns can be determined, and then the type of the overall probability distribution can be determined based on the distribution characteristics.

[0093] In order to intuitively present the numerical distribution of multiple first compatibilities, the embodiments of the present application are combined with the appendix of the specification. Figure 1b and 1c Make an introduction.

[0094] exist Figure 1b In , multiple first compatibilities show the characteristics of normal distribution. Therefore, it can be considered that the type of numerical distribution they present is normal distribution, that is, multiple first compatibilities obey normal distribution.

[0095] Normal distribution means that the probability distribution of a set of data or random variables conforms to the characteristics of normal distribution. The relevant descriptions of normal distribution are as follows: 1. Definition of normal distribution: Normal distribution is a symmetrical distribution. Its probability density function curve is bell-shaped, high in the middle, low on both sides, and symmetrical. Normal distribution has two parameters: mean ( μ ) and standard deviation ( σ ), which determine the center position and dispersion of the distribution respectively.

[0096] 2. Characteristics of normal distribution: Normal distribution has some important characteristics, such as the mean, median and mode are equal, all equal to μ ; Standard deviation σ Determines the dispersion of the distribution. σ The larger the value, the more dispersed the distribution; the normal distribution curve is about the mean μ symmetry.

[0097] 3. Why data follows a normal distribution: Many natural and social phenomena tend to follow a normal distribution, possibly due to the central limit theorem, which states that under appropriate conditions, the sum of a large number of independent random variables will approximately follow a normal distribution.

[0098] 4. Example: For example, in statistics, we often assume that people's height, weight and other physiological characteristics follow a normal distribution; in physics, error analysis often assumes that errors follow a normal distribution; in the financial field, stock price fluctuations are often assumed to follow a normal distribution (although this may not be the case in reality). In the embodiments of this application, the inventors have found through research that if the AI ​​model does not have AI hallucinations, the numerical distribution of multiple first compatibilities corresponding to multiple proper nouns in the prediction sentence output by the AI ​​model is also a normal distribution, or an approximate normal distribution.

[0099] It should be noted that, for the convenience of description, in the embodiments of the present application, "approximate normal distribution" is regarded as a special case of normal distribution and is included in the concept of "normal distribution".

[0100] like Figure 1c As shown, multiple first compatibilities obey an approximately normal distribution.

[0101] In an optional embodiment, the so-called "approximate normal distribution" may refer to the distribution curve about the mean μ Approximately symmetrical.

[0102] Figure 1b and Figure 1cOnly the normal distribution is taken as an example to intuitively present the situation where multiple first compatibilities obey a certain overall probability distribution. It can be understood by those skilled in the art that the type of numerical distribution presented by the multiple first compatibilities determined by executing step 13 may not be a normal distribution - as will be described later, in this case, it will be determined that the model has AI hallucinations.

[0103] After determining the numerical distributions of the multiple first compatibilities, it can be determined whether the model has AI hallucinations based on the type of the numerical distributions.

[0104] In an optional implementation, the features that characterize the AI ​​hallucination in the model can be preset from two aspects: "the size of the first compatibility" and "the type of numerical distribution presented by the multiple first compatibilities". Thus, in step 14, by comparing whether the types of the multiple first compatibilities and the overall probability distribution match the preset features, a corresponding judgment result can be obtained. Specifically, if they match, it is determined that the model has AI hallucination; if they do not match, it is determined that the model does not have AI hallucination.

[0105] In an optional implementation, the preset feature can be obtained by statistically analyzing the common features reflected in the prediction sentences output by the models when the AI ​​hallucination occurs. The models may include, for example: a model, and other models that perform the same natural language task as the model.

[0106] In an optional embodiment, the predetermined features capable of characterizing the AI ​​hallucination of the model may include: Feature 1: an average value of the absolute values ​​of the plurality of first compatibilities is less than a preset first threshold; Feature 2: The type of numerical distribution presented by multiple first compatibilities is not a normal distribution.

[0107] In the embodiment of the present application, the first threshold is a critical value for distinguishing whether the model has AI hallucination. Generally, if the average value of the absolute values ​​of the multiple first compatibilities is greater than or equal to the threshold, it indicates that the model has a high probability of not having AI hallucination; and if the average value is less than the threshold, it indicates that the model has a high probability of having AI hallucination. Exemplarily, the first threshold can be 0.8, for example.

[0108] In an optional implementation, the first threshold may be preset according to experience based on actual needs. In a specific example, for example, statistics may be collected on "several prediction statements that are confirmed to be output by the model when AI hallucination occurs", and the mean of the first compatibility of all proper nouns in the prediction statements to which they belong may be obtained, and the mean may be set as the first threshold.

[0109] In an optional implementation, "the multiple first compatibilities are greater than or equal to the first threshold" may mean: more than a set proportion of the multiple first compatibilities are greater than or equal to the first threshold; or, it may mean: the average of the multiple first compatibilities is greater than or equal to the first threshold.

[0110] Correspondingly, "the multiple first compatibilities are less than the first threshold" may mean: more than a set proportion of the multiple first compatibilities are less than the first threshold; or, it may mean: the average of the multiple first compatibilities is less than the first threshold.

[0111] As for the “type of numerical distribution presented by multiple first compatibilities”, generally speaking, if the model outputs the prediction statement without AI hallucination, then the distribution will theoretically conform to the normal distribution—that is, the number of proper nouns with very high or very low first compatibilities is small, while the number of proper nouns with medium first compatibilities is large. Therefore, in the embodiment of the present application, whether the model has AI hallucination is determined based on whether the multiple first compatibilities corresponding to the multiple proper nouns obey the normal distribution as a whole.

[0112] In an optional embodiment, the first compatibility of all proper nouns in the prediction statement can be input into a distribution statistical model to obtain the type of numerical distribution presented by the first compatibility of all proper nouns in the prediction statement output by the model, and then determine whether the type of the overall probability distribution is a normal distribution.

[0113] The distribution statistics model can be trained using "sentences that are manually labeled with types of overall probability distribution" as training samples.

[0114] For example, for a sentence that has been pre-confirmed that the type of the overall probability distribution conforms to the normal distribution, its "compatibility set consisting of the first compatibility degree of the proper noun in the sentence" can be labeled "type of normal distribution" as a positive sample in the training sample; for a sentence that has been pre-confirmed that the type of the overall probability distribution does not conform to the normal distribution, the corresponding compatibility set can be labeled "type of non-normal distribution" as a negative sample in the training sample.

[0115] With positive and negative samples as input and the type of numerical distribution presented by the first compatibility in the compatibility set as the prediction target, a distribution statistics model for identifying the type of numerical distribution presented by the first compatibility can be trained.

[0116] In an optional embodiment, when multiple first compatibilities have the above-mentioned "Feature 1", or the type of numerical distribution presented by multiple first compatibilities has the above-mentioned "Feature 2", it can be determined that the model has AI hallucinations; and when multiple first compatibilities do not have the above-mentioned "Feature 1", and the type of numerical distribution presented by multiple first compatibilities does not have the above-mentioned "Feature 2", it can be determined that the model does not have AI hallucinations.

[0117] Generally, if the AI ​​model does not have AI hallucination, the first compatibility of the proper nouns in the predicted sentence it outputs will be higher; at the same time, the numerical distribution presented by the first compatibilities of multiple proper nouns will also match the expected numerical distribution type. Based on the above characteristics, the embodiment of the present application uses "the first compatibilities of multiple proper nouns" and "the numerical distribution presented by multiple first compatibilities" as the basis for judging whether the AI ​​model has AI hallucination, thereby providing a solution for effectively identifying whether a large model has AI hallucination.

[0118] At the same time, since the basis for judging whether the model has AI hallucinations is refined to the "semantic element" dimension of the word, the judgment result obtained by using the method provided in the embodiment of the present application is more accurate.

[0119] In an embodiment of the present application, in order to avoid the first prediction statement outputted when the model has AI hallucination, which may cause problems such as misleading users who need to obtain the first prediction statement, when it is determined that the model has AI hallucination, an error correction operation on the first prediction statement may be further performed.

[0120] In an optional implementation, the following steps may be used to implement error correction of the first prediction statement: Step I: Determine the second word in the first prediction sentence; The compatibility between the semantic element of the second word and the semantic element of the third word is less than a second threshold, the second word and the third word belong to the first predicted sentence, and the third word is a proper noun. The second threshold may be preset.

[0121] By executing step I, it is possible to determine from the first predicted sentence a word (the second word) that is less compatible with the proper noun (the third word) in terms of semantic elements as a word to be corrected. A word (the second word) that is less compatible with the proper noun (the third word) in terms of semantic elements can also be considered as a word that is incompatible with the semantic elements of the proper noun.

[0122] In an optional implementation, the compatibility between the semantic element of the second word and the semantic element of the third word may be determined based on the semantic compatibility and part-of-speech compatibility between the second word and the third word (proper noun).

[0123] In a specific example, the compatibility of the semantic elements of the second word and the third word (proper noun) = semantic compatibility + part of speech compatibility; or, the compatibility of the semantic elements of the second word and the third word (proper noun) = Semantic compatibility+ Part-of-speech compatibility, where and is the weight.

[0124] Semantic compatibility can be calculated based on the semantic element vector of a word. Assuming that the semantic element vector of word A is {

semantic element 1

semantic element 2

semantic element 3

semantic element 4

semantic element 5

semantic element 1

semantic element 2

semantic element 3

semantic element 4

semantic element 5

[0125] Part-of-speech compatibility can be calculated using the following method: The word combination consisting of word A and word B is first input into a pre-trained model with the function of "obtaining the part-of-speech feature vector of the word combination" to obtain the part-of-speech feature vector combination corresponding to the word combination output by the pre-trained model; wherein the part-of-speech feature vector combination is composed of the part-of-speech feature vectors of each of word A and word B; Then, the part-of-speech feature vector is input into a pre-trained neural network with the function of "inferring the acceptability of word combinations", and the neural network infers the acceptability of the word combination as the part-of-speech compatibility between word A and word B.

[0126] Step II: Obtain the fourth word from the preset word library; The compatibility between the semantic element of the fourth word and the semantic element of the third word is greater than or equal to a third threshold.

[0127] By executing step II, it is possible to obtain a word (fourth word) that is relatively compatible with the proper noun (third word) in terms of semantic elements from the preset word library as a word for error correction.

[0128] Generally, the second threshold may be less than or equal to the third threshold.

[0129] Step III: Replace the second word in the first prediction sentence with the fourth word to obtain the second prediction sentence.

[0130] By executing step III, the word (second word) in the first prediction sentence that has a relatively low semantic compatibility with the proper noun (third word) can be replaced by a word (fourth word) that has a relatively high semantic compatibility with the proper noun (third word). As a result, compared with the first prediction sentence, the proper noun (third word) and other words in the second prediction sentence are more reasonable in appearing in the first prediction sentence, thereby correcting the unreasonable content of the first prediction sentence output by the model due to AI hallucination.

[0131] Based on the above principle, in a specific example, the following method can be used to correct the first prediction statement: First, words compatible with the semantic elements of the proper noun are obtained from the preset word library as compatible words; Then, the words in the first predicted sentence that are incompatible with the semantic elements of the proper noun are replaced with compatible words.

[0132] The preset word library is used to store words with the same or similar semantics as the proper nouns, and / or words with the same or similar parts of speech as the proper nouns. The word library here refers to a word library that is pre-built to match the semantic vector library in order to build a semantic vector library. Based on actual needs, the word library can be built according to the official dictionary of a certain language (such as the Modern Chinese Dictionary).

[0133] In the embodiment of the present application, the vocabulary built can be of different types. In a specific example, there can be a vocabulary with the same / similar semantics - every two words in the vocabulary satisfy the "graph relationship between words by adding or / and reducing a certain semantic element to become another word"; there can also be a vocabulary with the same / similar part of speech - every two words in the vocabulary satisfy the "graph relationship between words by adding or / and reducing a certain grammatical semantic element to become another word".

[0134] In the embodiments of the present application, words having the same or similar semantics as a proper noun, and words having the same or similar parts of speech as a proper noun, are generally words that are compatible with the semantic elements of the proper noun.

[0135] The words incompatible with the semantic elements of the proper nouns in the first prediction sentence (such as the second word mentioned above) are incompatible with the proper nouns in terms of semantic elements and part of speech. In other words, the proper nouns and such words appear in the same first prediction sentence, which is very abrupt and unreasonable, indicating that the model is very likely to output the first prediction sentence when AI hallucination occurs. However, compatible words (such as the fourth word mentioned above) are compatible with the proper nouns in terms of semantic elements and part of speech. In other words, it is more reasonable and will not appear abrupt for the proper nouns and compatible words to appear in the same first prediction sentence at the same time. Such a sentence is highly unlikely to be the first prediction sentence output when the model has AI hallucinations.

[0136] Therefore, by using compatible words to replace words incompatible with the semantic elements of the proper noun in the first prediction sentence, it is possible to correct the originally unreasonable prediction sentence into a reasonable prediction sentence, thereby achieving error correction of the prediction sentence.

[0137] In an optional implementation, a word compatible with the semantic element of the proper noun is obtained from a preset word library as a compatible word (such as the fourth word mentioned above), which can be specifically implemented in the following manner: Step i: Obtain the fifth word from the preset word library; The fifth word is also a word that is incompatible with the semantic element of the proper noun. For example, the compatibility between the semantic element of the fifth word and the semantic element of the third word is less than the second threshold.

[0138] Step II: determine the first semantic element included in the fifth word; The first semantic element refers to a semantic element whose compatibility with the semantic element of the third word is greater than or equal to a third threshold.

[0139] In step ii, all the first sense elements included in the fifth word may be determined, or part (for example, one) of the first sense elements included in the fifth word may be determined.

[0140] Step III: Obtain the fourth word composed of the first semantic element from the preset vocabulary.

[0141] The fourth word can be used as the compatible word mentioned above to correct the first predicted sentence.

[0142] After correcting the first predicted sentence, a corrected text may be obtained.

[0143] In an optional implementation, for the error correction text, the first compatibility of each proper noun can be recalculated to obtain multiple recalculated first compatibilities; and it is determined whether the average of the absolute values ​​of the multiple recalculated first compatibilities is greater than or equal to a first threshold, and it is determined whether the numerical distribution presented by the multiple recalculated first compatibilities is a normal distribution; if both are yes, the error correction text is determined to be qualified.

[0144] Furthermore, in order to improve the performance of the model and reduce the probability of AI hallucinations in the model, qualified error correction texts can be used as training samples for training the model. Subsequently, the model can be retrained based on the training samples including qualified error correction texts.

[0145] Since the qualified error correction text still retains the proper nouns in the original first prediction sentence, and is corrected to the average of the absolute values ​​of multiple recalculated first compatibilities being greater than or equal to the first threshold, the qualified error correction text has the dual attributes of "retaining the core semantics of the first prediction sentence" and "not having the characteristics that characterize the model as having AI hallucinations". Based on such an error correction text retraining model, the probability of AI hallucinations in the retrained model can be effectively reduced without reducing the original performance of the model.

[0146] In an optional implementation, the training samples (first training samples) used to train the model may also be processed to improve the reasoning performance of the model.

[0147] Specifically, after determining that the model has AI hallucinations, you can perform the following operations: Step ⑴: Obtain the sixth word from the preset word library; The compatibility between the semantic element of the sixth word and the semantic element of the proper noun in the first predicted sentence is less than the third threshold.

[0148] As mentioned above, the preset word library is used to store: words with the same or similar semantics as the proper noun, and / or words with the same or similar parts of speech as the proper noun.

[0149] By executing step (1), it is possible to obtain words that are incompatible with the semantic elements of the proper noun from the preset word library. Such words can be called incompatible words.

[0150] Step (2): based on the proper noun and the sixth word, determining first sample data in the first training sample, and performing a masking operation on the first sample data to obtain second sample data; The first sample data is: sample data (specifically, a sentence) including the proper noun in the first predicted sentence and the sixth word obtained from the preset word library.

[0151] Mask, that is, mask, is also called mask. In the embodiment of the present application, performing a mask operation on the first sample data refers to blocking a portion of the content of the first sample data. In a specific example, if the first sample data includes: "Jintian Shigelang Saturday, let's have dinner together", the proper noun is "Saturday", and the sixth word includes "Jintian Shigelang", then the second sample data obtained after masking the first sample data is "[mask], let's have dinner together". The [mask] therein indicates that the first sample data is blocked.

[0152] By performing the above masking operation on the first sample data, the model can achieve memory fuse for the masked words. Based on this, when the model is subsequently trained using the masked training samples, the model can be forced to learn richer context information, thereby improving the generalization ability of the model.

[0153] Step (3): Use the second training sample to train the model, wherein the second training sample includes: a second prediction statement and second sample data.

[0154] The following examples mainly introduce a specific implementation method of applying the above model detection method provided in the examples of the present application to actual scenarios.

[0155] The actual scenario is a loan collection scenario. The specific architecture diagram of this scenario is as follows: Figure 2 As shown, it includes: the user terminal of the conversation participant A and the machine agent server.

[0156] Among them, a machine agent is running on the user terminal, and the conversation participant A can input the questions he wants to ask into the machine agent in text format; the machine agent can send the received text to the machine agent server.

[0157] The machine agent server is deployed with LLM, which can predict the corresponding answer text based on the received text and feed it back to the user terminal of the interlocutor A.

[0158] In this scenario, the implementation process of the model detection method provided in the embodiment of the present application is as follows: Figure 3 As shown, the following steps are included: Step 31: The interlocutor A inputs the text as a question into the machine agent, and the machine agent sends the text to the machine agent server; Step 32: The machine agent server obtains the answer text output by the LLM for the text as the pre-answer text; The LLM is equivalent to the model described in the above embodiment. The pre-answer text is equivalent to the first prediction sentence described in the above embodiment.

[0159] Step 33: The machine agent server obtains proper nouns in the pre-answer text, and other words (common words) except proper nouns; Step 34: The machine agent server obtains the semantic vector of each proper noun and common word; Step 35: The machine agent server calculates the first compatibility of each proper noun based on the semantic element vectors of each proper noun and the common word; Step 36: The machine agent server determines whether the pre-answer text is a qualified text; Specifically, whether the pre-answer text is a qualified text can be determined by comparing a preset first threshold with the first compatibility of the proper nouns and determining whether the numerical distribution of the first compatibility of all proper nouns conforms to a normal distribution.

[0160] Step 37: If the comparison result is that the average value of the absolute value of the first compatibility of the proper noun in the pre-answer text is greater than or equal to the first threshold, and the numerical distribution of the first compatibility conforms to the normal distribution, the machine agent server determines that the pre-answer text is qualified, and thus feeds the pre-answer text back to the interlocutor A as the final answer text; Step 38: If the average absolute value of the first compatibility of the proper noun in the pre-answer text is less than the first threshold, or the numerical distribution of the first compatibility does not conform to the normal distribution, the machine agent server determines that the pre-answer text is unqualified, and the machine agent server replies to the interlocutor A that "the model cannot reply"; Step 39: The machine agent server obtains words that are incompatible with the semantic elements of the proper noun from the preset word library, and performs a masking operation on the sentences containing the obtained words and the proper noun in the training samples used for training the LLM; Step 310: the machine agent server corrects the pre-answer text until it is corrected into a qualified text, and then adds the qualified text to the training sample set for training LLM (the training sample set after performing the above masking operation) as a new training sample; Step 311: The machine agent server iteratively trains the LLM based on the training sample set, and reloads the trained LLM after the LLM training is completed.

[0161] The training sample set is a set of pre-answer texts to which texts corrected as qualified are added, and a set of training samples to which the aforementioned masking operation has been performed.

[0162] At this point, the machine agent server has achieved the optimization of LLM. By optimizing LLM in this way, the probability of LLM experiencing AI hallucinations can be reduced. When LLM faces the questions that speaker A wants to ask again in the future, it is likely to provide a reasonable answer.

[0163] In order to solve the problem in the prior art of how to identify whether a large model has AI hallucination, based on the same inventive concept as the above-mentioned embodiments of the present application, the embodiments of the present application also provide a model detection device.

[0164] The specific structural diagram of the device is shown in Figure 4 As shown, it includes the following functional units: An acquisition unit 41 is used to acquire a first prediction sentence output by the model, and determine a plurality of proper nouns included in the first prediction sentence; a compatibility determination unit 42, configured to determine a first compatibility of each of the proper nouns to obtain a plurality of first compatibilities, wherein the first compatibility is used to characterize the degree of compatibility between the semantic element of the proper noun and the semantic element of the first word, wherein the first word and the proper noun belong to the first predicted sentence; The detection result determination unit 43 is used to determine whether the model has artificial intelligence hallucination based on the multiple first compatibilities and the numerical distribution presented by the multiple first compatibilities.

[0165] In an optional implementation, the detection result determination unit 43 may be specifically used to: if the average value of the absolute values ​​of the multiple first compatibilities is less than a first threshold, and the numerical distribution presented by the multiple first compatibilities is not a normal distribution, then determine that the model exhibits artificial intelligence hallucination.

[0166] In an optional implementation, the device provided by the system of the embodiment of the present application may further include: a word determination unit, configured to determine a second word in the first predicted sentence, wherein the compatibility between a semantic element of the second word and a semantic element of a third word is less than a second threshold, the second word and the third word belong to the first predicted sentence, and the third word is a proper noun; A word acquisition unit, configured to acquire a fourth word from a preset word library, wherein the compatibility of the semantic element of the fourth word with the semantic element of the third word is greater than or equal to a third threshold; A replacement unit is used to replace the second word in the first prediction sentence with a fourth word to obtain a second prediction sentence.

[0167] In an optional implementation, the word acquisition unit may be specifically used for: Acquire a fifth word from the preset word library, wherein the compatibility between the semantic element of the fifth word and the semantic element of the third word is less than the second threshold; Determine all first semantic elements included in the fifth word, wherein the compatibility of the first semantic elements with the semantic elements of the third word is greater than or equal to the third threshold; A fourth word consisting of all the first semantic elements is obtained from the preset word library.

[0168] In an optional embodiment, the compatibility determination unit 42 can be specifically used to: determine the first compatibility of the proper noun based on the second compatibility and the third compatibility of the proper noun; the second compatibility is used to characterize the degree of compatibility between the semantic elements of the proper noun and the semantic elements of the first word; the third compatibility is used to characterize the degree of compatibility between the part of speech of the proper noun and the part of speech of the first word.

[0169] In an optional embodiment, the model is trained based on a first training sample. In this case, the word acquisition unit can also be used to: if it is determined that the model has artificial intelligence hallucinations, obtain a sixth word from a preset vocabulary, and the compatibility of the semantic elements of the sixth word with the semantic elements of the proper noun is less than a third threshold. The preset vocabulary is used to store: words with the same or similar semantics as the proper noun, and / or words with the same or similar parts of speech as the proper noun.

[0170] Furthermore, the device provided in the embodiment of the present application may also include: a masking unit, configured to determine first sample data in the first training sample based on the proper noun and the sixth word, and perform a masking operation on the first sample data to obtain second sample data; the first sample data includes the proper noun and the sixth word; A training unit is used to train the model using a second training sample; the second training sample includes: a second prediction statement, and the second sample data.

[0171] The at least one model detection device used in the embodiment of the present application can achieve the following beneficial effects: If the AI ​​model does not have AI hallucination, the first compatibility of the proper nouns in the predicted sentence it outputs will be higher; at the same time, the numerical distribution presented by the first compatibilities of multiple proper nouns will also match the expected numerical distribution type. Based on the above characteristics, the embodiment of the present application uses "the first compatibility of multiple proper nouns" and "the numerical distribution presented by multiple first compatibilities" as the basis for judging whether the AI ​​model has AI hallucination, thereby providing a solution for effectively identifying whether a large model has AI hallucination.

[0172] Based on the same inventive concept as the aforementioned embodiments of the present application, the embodiments of the present application also provide a computing device to solve the problem existing in the prior art of how to identify whether a large model has AI hallucination.

[0173] like Figure 5 As shown, the computing device includes: a memory 51 and a processor 52. The memory 51 can be configured to store various other data to support operations on the electronic device. Examples of these data include instructions for any application or method for operating on the electronic device. The memory 51 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.

[0174] The processor 52 is coupled to the memory 51 and is used to execute the computer program stored in the memory 51 to execute the model detection method or text error correction method provided in the embodiment of the present application.

[0175] When the processor 52 executes the computer program in the memory 51, in addition to the above functions, it can also realize other functions, which can be seen from the description of the previous embodiments for details.

[0176] Further, such as Figure 5 As shown, the computing device also includes: a display 54, a communication component 53, a power component 55, an audio component 56 and other components. Figure 5 Only some components are shown schematically, and it does not mean that the computing device only includes Figure 5 Components shown.

[0177] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a computer, the methods provided in the above embodiments can be implemented.

[0178] Accordingly, an embodiment of the present application further provides a computer program product, which stores instructions, and when the instructions are executed by a computer, the computer implements the methods provided in the above embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0179] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A model detection method, characterized in that: include: Acquire a first prediction sentence output by the model, and determine a plurality of proper nouns included in the first prediction sentence; Determining a first compatibility of each of the proper nouns to obtain a plurality of first compatibility degrees, wherein the first compatibility degrees are used to characterize the degree of compatibility between a semantic element of the proper noun and a semantic element of a first word, wherein the first word and the proper noun both belong to the first predicted sentence; Based on the multiple first compatibilities and the numerical distributions presented by the multiple first compatibilities, it is determined whether the model exhibits artificial intelligence hallucinations.

2. The method according to claim 1, characterized in that The determining whether the model has artificial intelligence hallucination based on the multiple first compatibilities and the numerical distributions presented by the multiple first compatibilities includes: If the average value of the absolute values ​​of the multiple first compatibilities is less than a first threshold, and the numerical distribution presented by the multiple first compatibilities is not a normal distribution, it is determined that the model has an artificial intelligence hallucination.

3. The method according to claim 1, characterized in that: After determining that the model exhibits artificial intelligence hallucinations, the method further includes: determining a second word in the first predicted sentence, wherein a compatibility degree between a semantic element of the second word and a semantic element of a third word is less than a second threshold, the second word and the third word belong to the first predicted sentence, and the third word is a proper noun; Acquire a fourth word from a preset word library, wherein the compatibility of the semantic element of the fourth word with the semantic element of the third word is greater than or equal to a third threshold; The second word in the first predicted sentence is replaced by a fourth word to obtain a second predicted sentence.

4. The method according to claim 3, characterized in that The acquiring the fourth word from the preset word library includes: Acquire a fifth word from the preset word library, wherein the compatibility between the semantic element of the fifth word and the semantic element of the third word is less than the second threshold; Determine a first semantic element included in the fifth word, wherein the compatibility of the first semantic element with the semantic element of the third word is greater than or equal to the third threshold; A fourth word composed of the first semantic element is obtained from the preset word library.

5. The method according to claim 1, characterized in that The determining the first compatibility of the proper noun comprises: For each proper noun, the first compatibility of the proper noun is determined based on the second compatibility and the third compatibility of the proper noun; the second compatibility is used to characterize the degree of compatibility between the semantic elements of the proper noun and the semantic elements of the first word; the third compatibility is used to characterize the degree of compatibility between the part of speech of the proper noun and the part of speech of the first word.

6. The method according to claim 3 or 4, characterized in that The model is obtained by training based on the first training sample, and the method further includes: If it is determined that the model has an artificial intelligence hallucination, a sixth word is obtained from a preset word library, the compatibility of the semantic element of the sixth word with the semantic element of the proper noun is less than a third threshold, and the preset word library is used to store words with the same or similar semantics as the proper noun, and / or words with the same or similar parts of speech as the proper noun; Based on the proper noun and the sixth word, determining first sample data in the first training sample, the first sample data including the proper noun and the sixth word; Performing a mask operation on the first sample data to obtain second sample data; The model is trained using a second training sample, where the second training sample includes: a second prediction statement and the second sample data.

7. A model detection device, characterized in that: include: an acquisition unit, configured to acquire a first prediction sentence output by the model, and determine a plurality of proper nouns included in the first prediction sentence; a compatibility determination unit, configured to determine a first compatibility of each of the proper nouns to obtain a plurality of first compatibilities, wherein the first compatibility is used to characterize the degree of compatibility between the semantic element of the proper noun and the semantic element of a first word, wherein the first word and the proper noun belong to the same first predicted sentence; A detection result determination unit is used to determine whether the model has artificial intelligence hallucination based on the multiple first compatibilities and the numerical distribution presented by the multiple first compatibilities.

8. A computing device, characterized in that include: Memory and processor, wherein: The memory is used to store computer programs; The processor is coupled to the memory and is used to execute the computer program stored in the memory to execute the method described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer, can implement the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product stores instructions, and when the instructions are executed by a computer, the computer implements the method according to any one of claims 1 to 6.

Images