A method, system, device and medium for multi-dimensional recognition model hallucination

Abstract

The application discloses a method, system, device and medium for multi-dimensional hallucination identification of a model, which generates an initial reply corresponding to a question and answer request in response to a target model, and acquires the initial reply of the target model. Target entities, target entity attributes of the target entities and correlation of the target entities are extracted from the initial reply. Based on the target entities, the target entity attributes of the target entities and the correlation of the target entities, the initial reply is subjected to multi-dimensional hallucination evaluation to obtain an initial evaluation result corresponding to each dimension in multi-dimensions. The multi-dimensional hallucination evaluation includes fact evaluation, logic evaluation and scene evaluation. According to the initial evaluation results, a hallucination identification result of the target model is generated. The hallucination can be identified by multi-dimensional evaluation through entity extraction and combined with multi-dimensional evaluation results, multi-dimensional evaluation of fact authenticity, logic integrity and scene adaptability of the model output content is realized, and the hallucination type is accurately identified and located.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a multi-dimensional identification model hallucination method, system, device and medium. BACKGROUND

[0002] Large language models (LLMs) are widely used in single-modal text dialogue scenarios, such as medical field question and answer dialogue, e-commerce field customer service consultation, education field knowledge point answering, etc., which can efficiently generate natural language responses. However, LLMs generally have the problem of "hallucination", i.e. the generated content is factually incorrect, logically inconsistent or irrelevant, which may cause safety risks in critical fields such as medicine, and affect service quality and information accuracy in e-commerce and education fields. The root causes include training data bias, model overfitting and lack of factual support.

[0003] Current existing hallucination identification methods rely on external knowledge bases or labeled data, and their identification effect is limited by the completeness and real-time updating ability of the knowledge base. Moreover, they can only identify specific types of hallucinations or adapt to a single scene, and are prone to problems such as data unavailability, privacy leakage and poor scalability. Moreover, they lack multi-scene adaptability and cannot match the different tolerance and identification needs of hallucinations in different fields such as medicine and finance, making it difficult to meet the efficient and accurate identification needs in actual applications, resulting in limited actual application effect. SUMMARY

[0004] The following is a summary of the subject matter described in detail in this document. This summary is not intended to limit the scope of protection of the claims.

[0005] The main purpose of the embodiments of the present disclosure is to propose a multi-dimensional identification model hallucination method, system, device and storage medium, which can perform multi-dimensional evaluation through entity extraction, and identify hallucinations combined with multi-dimensional evaluation results, realizing multi-dimensional evaluation of the factual truth, logical integrity and scene adaptability of the model output content, and accurate identification and positioning of hallucination types.

[0006] The first aspect of the embodiments of the present application provides a multi-dimensional identification model hallucination method, for a central controller, the method comprising:

[0007] generating a corresponding initial reply in response to a target model responding to a question and answer request, obtaining the initial reply of the target model;

[0008] extracting a target entity, a target entity attribute of the target entity and an association relationship of the target entity from the initial reply; the target entity is extracted based on text disassembly of the initial reply into multiple text segments;

[0009] perform multi-dimensional illusion evaluation on the initial reply based on the target entity, the target entity attribute of the target entity, and the association relationship of the target entity, to obtain an initial evaluation result corresponding to each dimension in the multi-dimensions; the multi-dimensional illusion evaluation includes fact evaluation, logic evaluation, and scene evaluation; the fact evaluation is quantitative evaluation based on factual content associated with the target entity; the logic evaluation is quantitative evaluation based on a logic rule associated with the target entity attribute and the entity association relationship; and the scene evaluation is quantitative evaluation based on scene relevance associated with the target entity and the target entity attribute.

[0010] generate an illusion recognition result of the target model according to the initial evaluation results.

[0011] In some embodiments of the present application, the extracting of the target entity, the target entity attribute of the target entity, and the association relationship of the target entity from the initial reply includes:

[0012] perform text disassembly on the initial reply to obtain a plurality of text segments.

[0013] perform word segmentation, part-of-speech tagging, and named entity recognition on each of the text segments, to obtain a word segmentation result of each of the text segments, part-of-speech tagging information of each of the text segments, and a target entity of each of the text segments.

[0014] analyze attribute content corresponding to the target entity and association relationship between the target entities based on the word segmentation result of each of the text segments and the part-of-speech tagging information of each of the text segments, to obtain the target entity attribute and the entity association relationship.

[0015] In some embodiments of the present application, before the performing of the multi-dimensional illusion evaluation on the initial reply based on the target entity, the target entity attribute of the target entity, and the association relationship of the target entity, to obtain an initial evaluation result corresponding to each dimension in the multi-dimensions, the method further includes: determining evaluation accuracy of the multi-dimensional illusion evaluation according to an application scenario of the question and answer request; the application scenario of the question and answer request includes one of the following: a real-time response scenario and an offline response scenario, and the evaluation accuracy of the offline response scenario is greater than the evaluation accuracy of the real-time response scenario.

[0016] In some embodiments of the present application, the performing of the multi-dimensional illusion evaluation on the initial reply based on the target entity, the target entity attribute of the target entity, and the association relationship of the target entity, to obtain an initial evaluation result corresponding to each dimension in the multi-dimensions, includes:

[0017] dissolving the initial reply into multiple fact units based on the target entity, the target entity attribute, and the entity association relationship; the fact unit represents fact content of the initial reply, and the fact unit includes a fact entity, a fact attribute, and a fact attribute value;

[0018] According to the application scenario of the initial reply, a knowledge graph corresponding to the initial reply and a fact rule set corresponding to the initial reply are called;

[0019] The matching degrees of the fact units and the knowledge graph corresponding to the initial reply are calculated.

[0020] According to the fact rule set corresponding to the question and answer request, rule verification is performed on each fact unit to obtain a corresponding verification score.

[0021] According to the matching degrees of the fact units and the verification scores of the fact units, an initial evaluation result corresponding to the fact evaluation is calculated.

[0022] In some embodiments of the present application, the multi-dimensional hallucination evaluation of the initial reply based on the target entity, the target entity attribute of the target entity, and the association relationship of the target entity to obtain an initial evaluation result corresponding to each dimension in the multi-dimension is further included.

[0023] Based on the target entity attribute and the entity association relationship, a hierarchical logical network of the initial reply is constructed; the hierarchical logical network includes a sentence-level logical chain and a paragraph-level logical chain, the sentence-level logical chain represents a mapping relationship between the target entity and the target entity attribute in each text segment in the initial reply, and the paragraph-level logical chain represents a connection logic of the target entity association relationship between the text segments in the initial reply.

[0024] A preset logical rule library is called to perform local logical detection and global logical detection on the hierarchical logical network respectively to obtain a local detection result and a global detection result; the local logical detection is used to detect the sentence-level logical chain, and the global logical detection is used to detect the paragraph-level logical chain.

[0025] According to the local detection result and the global detection result, an initial evaluation result corresponding to the logical evaluation is calculated.

[0026] In some embodiments of the present application, the multi-dimensional hallucination evaluation of the initial reply based on the target entity, the target entity attribute of the target entity, and the association relationship of the target entity to obtain an initial evaluation result corresponding to each dimension in the multi-dimension is further included.

[0027] Based on the application scenario of the question-and-answer request, the corresponding scenario feature library is retrieved. The scenario feature library includes a scenario entity set, a scenario keyword set, and scenario-independent identifiers.

[0028] Based on the scenario feature library, calculate the entity matching rate, keyword coverage rate, and proportion of irrelevant content in the initial response;

[0029] By integrating the entity matching rate, the keyword coverage rate, and the proportion of irrelevant content, the initial evaluation result corresponding to the scenario evaluation is obtained.

[0030] In some embodiments of this application, generating the hallucination recognition result of the target model based on the initial evaluation results includes:

[0031] By integrating the initial evaluation results of the factual assessment, the initial evaluation results of the logical assessment, and the initial evaluation results of the scenario assessment, the target evaluation result is obtained.

[0032] Based on the application scenario of the question-and-answer request, the corresponding hallucination threshold is invoked;

[0033] The target evaluation result is compared with the hallucination threshold to obtain the comparison result;

[0034] If the target evaluation result is higher than or equal to the hallucination threshold, a hallucination recognition result without hallucination is generated; if the target evaluation result is lower than the hallucination threshold, an initial recognition result with hallucination is generated, and the hallucination type of the initial response is located according to each initial evaluation result; the hallucination type is determined based on each initial evaluation result and the hallucination evaluation threshold of the corresponding dimension; the hallucination recognition result is obtained by combining the initial recognition result and the hallucination type of the initial response.

[0035] To achieve the above objective, a second aspect of the present invention provides a system for multi-dimensional identification of model hallucinations, the system comprising:

[0036] The acquisition module is used to acquire the initial response of the target model in response to the question-and-answer request and generate the corresponding initial response.

[0037] The extraction module is used to extract the target entity, the target entity attributes, and the association relationships of the target entity from the initial response; the target entity is extracted after the initial response is decomposed into multiple text fragments.

[0038] The evaluation module is configured to perform multi-dimensional illusion evaluation on the initial reply based on the target entity, the target entity attribute of the target entity, and the association relationship of the target entity, to obtain an initial evaluation result corresponding to each dimension in the multi-dimensions; the multi-dimensional illusion evaluation includes fact evaluation, logic evaluation, and scene evaluation; the fact evaluation is quantitative evaluation based on fact content associated with the target entity; the logic evaluation is quantitative evaluation based on a logic rule associated with the target entity attribute and the entity association relationship; and the scene evaluation is quantitative evaluation based on scene relevance associated with the target entity and the target entity attribute.

[0039] The result module is configured to generate an illusion recognition result of the target model according to the initial evaluation results.

[0040] To achieve the above object, a third aspect of the embodiment of the present application provides an electronic device, which comprises at least one control processor and a memory connected in communication with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the method for multi-dimensional recognition model illusion.

[0041] To achieve the above object, a fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores computer executable instructions for causing a computer to perform the method for multi-dimensional recognition model illusion.

[0042] The embodiment of the present application provides a method for multi-dimensional recognition model illusion, which generates an initial reply corresponding to a target model in response to a question and answer request, obtains an initial reply of the target model, extracts a target entity, a target entity attribute of the target entity, and an association relationship of the target entity from the initial reply, performs multi-dimensional illusion evaluation on the initial reply based on the target entity, the target entity attribute of the target entity, and the association relationship of the target entity to obtain an initial evaluation result corresponding to each dimension in the multi-dimensions, the multi-dimensional illusion evaluation includes fact evaluation, logic evaluation, and scene evaluation, and generates an illusion recognition result of the target model according to the initial evaluation results, which can perform multi-dimensional evaluation through entity extraction, recognize illusion in combination with multi-dimensional evaluation results, realize multi-dimensional evaluation of fact authenticity, logic integrity, and scene adaptability of model output content, and accurately recognize and locate illusion types.

[0043] It can be understood that the beneficial effects of the above-mentioned second aspect to fourth aspect and related technologies are the same as the beneficial effects of the above-mentioned first aspect and related technologies, which can be referred to the related description in the first aspect, and will not be repeated here. Attached Figure Description

[0044] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0045] Figure 1 This is a flowchart illustrating a method for multi-dimensional identification of model hallucinations provided in an embodiment of this application;

[0046] Figure 2 This is a schematic diagram of the structure of a system for multi-dimensional identification of model hallucinations provided in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0048] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0049] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0050] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0051] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0052] Large language models (LLMs) are widely used in single-modal text conversation scenarios, such as medical field question and answer dialogues, e-commerce field customer service consultations, and education field knowledge point question and answer, and can efficiently generate natural language responses. However, LLMs generally have the problem of "hallucination", that is, the generated content is factually incorrect, logically inconsistent, or irrelevant, which may cause safety risks in critical fields such as medicine, and affect service quality and information accuracy in e-commerce and education fields. The root causes include training data bias, model overfitting, and lack of factual support.

[0053] Current existing hallucination identification methods rely on external knowledge bases or labeled data, and their identification effect is limited by the completeness and real-time updating ability of the knowledge base. Moreover, they can only identify specific types of hallucinations or adapt to a single scene, and are prone to problems such as data unavailability, privacy leakage, and poor scalability. Moreover, they lack multi-scene adaptability and cannot match the different tolerance and identification needs of hallucinations in medical, financial, and other fields, making it difficult to meet the efficient and accurate identification needs in actual applications, resulting in limited actual application effect.

[0054] Based on this, the embodiment of the present application provides a method, system, electronic device and medium for identifying hallucinations of a multi-dimensional recognition model, which aims to evaluate multiple dimensions through entity extraction, and then identify hallucinations based on the multi-dimensional evaluation results, thereby achieving multi-dimensional evaluation of the factual truthfulness, logical integrity, and scene adaptability of the model output content, and accurate identification and positioning of hallucination types.

[0055] The method, system, electronic device and medium for identifying hallucinations of a multi-dimensional recognition model provided by the embodiment of the present application are specifically explained by the following embodiments. First, the method for identifying hallucinations of a multi-dimensional recognition model in the embodiment of the present application is described.

[0056] The embodiment of the present application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (Artificial Intelligence, AI) is the use of digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Theory, method, technology and application system.

[0057] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc. Several major directions.

[0058] The method for multi-dimensional identification model hallucination provided in the embodiments of the present application relates to the technical field of artificial intelligence. The method for multi-dimensional identification model hallucination provided in the embodiments of the present application can be applied to a terminal, can also be applied to a server end, and can also be software running in the terminal or the server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server end can be configured as a separate physical server, can also be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform; and the software can be an application for implementing the method for multi-dimensional identification model hallucination, etc., but is not limited to the above forms.

[0059] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0060] It should be noted that in each specific embodiment of the present application, when relevant processing needs to be performed according to user information, user behavior data, user historical data, and user location information, etc., which are related to the identity or characteristics of the user, the user's permission or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the separate permission or separate consent of the user will be obtained through a pop-up window or a jump to a confirmation page, etc., and after obtaining the separate permission or separate consent of the user, the necessary user-related data for enabling the embodiments of the present application to normally operate will be obtained.

[0061] For this purpose, reference is made to Figure 1The embodiment of the application provides a multi-dimensional illusion identification method for a model, which is applied to a central controller, which can be a server, an electronic device, a mobile terminal or the like, and the method comprises the following steps S110-S140.

[0062] In step S110, an initial reply corresponding to the target model is generated in response to a question and answer request, and the initial reply of the target model is obtained.

[0063] In step S120, a target entity, a target entity attribute of the target entity and a correlation between the target entities are extracted from the initial reply; the target entity is extracted after text disassembly of the initial reply into multiple text segments.

[0064] In step S130, the initial reply is evaluated in multiple dimensions based on the target entity, the target entity attribute of the target entity and the correlation between the target entities, and an initial evaluation result corresponding to each dimension in the multiple dimensions is obtained; the multi-dimensional illusion evaluation includes fact evaluation, logic evaluation and scene evaluation; the fact evaluation is quantitative evaluation based on factual content associated with the target entity; the logic evaluation is quantitative evaluation based on logic rules associated with the target entity attribute and the correlation between the entities; and the scene evaluation is quantitative evaluation based on scene relevance associated with the target entity and the target entity attribute.

[0065] In step S140, an illusion identification result of the target model is generated according to the initial evaluation results.

[0066] In this step, when the target model generates an initial reply according to the received question and answer request, the initial reply generated by the target model is obtained. The target model can be any large language model. Then the initial reply is disassembled into multiple text segments, and the target entity, the target entity attribute and the correlation between the target entities are extracted from the initial reply.

[0067] Specifically, the target entity refers to a noun component with actual meaning extracted after text disassembly of the initial reply, which is preferably realized by using word segmentation, named entity recognition technology and dependency syntax analysis, and the core object is identified by disassembling the text segment to support subsequent multi-dimensional evaluation; the target entity attribute refers to feature or state information of the target entity, which is preferably extracted by using part-of-speech tagging and dependency relation analysis to extract entity modifiers or attribute values for analyzing the rationality of entity details; and the correlation between the target entities refers to logical or semantic connection between the target entities, which is preferably realized by using co-occurrence analysis, semantic role labeling or knowledge graph relationship reasoning for detecting whether the interaction between the entities conforms to the logic rules.

[0068] Further, based on the extracted target entity, target entity attribute and association relationship, multi-dimensional hallucination evaluation is performed on the initial reply, specifically including fact evaluation, logic evaluation and scene evaluation. Among them, the fact evaluation refers to the quantitative verification of the entity related facts based on the knowledge graph matching, preferably adopting the entity linking technology combined with the knowledge base retrieval to calculate the matching degree, solving the hallucination problem caused by factual errors; the logic evaluation refers to detecting the logic consistency at the sentence and paragraph level by constructing a hierarchical logic network, preferably adopting the logic rule library to verify the local and global logic chain to identify the hallucination caused by logic contradiction or break; the scene evaluation refers to calculating the relevance of the reply and the scene combined with the application scene characteristics, preferably through scene entity matching, keyword coverage rate and irrelevant content proportion analysis to solve the content off-topic or scene deviation problem. The fact evaluation is quantitatively evaluated based on the fact content related to the target entity. The logic evaluation is quantitatively evaluated based on the logic rules of the target entity attribute and entity association relationship. The scene evaluation is quantitatively evaluated based on the scene relevance of the target entity and attribute.

[0069] Further, according to the initial evaluation results of each dimension, the hallucination recognition result of the target model is generated, which can realize comprehensive analysis of the factuality, logic and scene relevance of the initial reply through multi-dimensional evaluation, thereby identifying possible hallucination problems, and further realizing a self-consistent evaluation system through in-depth analysis of the model output itself without relying on a complete external knowledge base, having strong field adaptability, and being able to be quickly deployed in different scenes such as medical treatment and finance to meet the differentiated hallucination recognition needs.

[0070] In an embodiment, for the scene of single-modal text dialogue, a multi-modal recognition model hallucination method is implemented to construct a large model hallucination recognition system. Specifically, the specific identification steps of the large model hallucination recognition system in the embodiment are as follows:

[0071] Step 1: Preprocess the initial question and the preliminary answer of the question by the LLM (the large model can be open source or closed source) to prepare for subsequent hallucination recognition. Among them, the preprocessing is the "text purification and structuring entrance" of the large model hallucination recognition system, and its core goal is to convert the original, unstructured user question and LLM response into clean, ordered and rich semantic information intermediate data, providing accurate input for subsequent semantic entropy calculation (logic consistency analysis), available entropy calculation (token level probability association) and sample entropy calculation (entity authenticity verification).

[0072] Specifically, the preprocessing includes text cleaning, text disassembly and key information extraction operations, and output standardized intermediate data. The detailed contents of the preprocessing in the embodiment are specifically introduced as follows:

[0073] First, text cleaning is used to eliminate noise and unify formats. Because the original text (especially user input questions) often contains redundant information, format chaos or invalid content, if directly entering subsequent processing, it will lead to errors in word segmentation, entity misidentification, and ultimately affect the accuracy of entropy calculation, so it is necessary to solve the problems of "noise interference" and "format inconsistency" to output "clean text". Specifically, it includes: removing redundant content that does not affect semantics (such as special symbols, meaningless spaces); correcting text errors (such as misspelling, pinyin / Chinese character mix); unifying text format (such as standardizing dates, numerical values, and units); filtering low-value content (such as pure mood words, repetitive expressions). At the same time, specific scenarios need to be optimized. For example, in the medical field: keep professional symbols such as "negative, positive, mmol / L", and prohibit filtering of key qualitative words such as "suspected, diagnosed"; in the financial field: standardize numerical formats such as "interest rate 3.5%" and "yield rate 2.8%", and avoid confusion between "3.5" and "3.5%"; in the education field: keep "BC, AD, formula symbols (such as π, √)", and correct term errors such as "Pythagorean theorem" to "Pythagorean theorem".

[0074] Secondly, the text is disassembled and the key information is extracted. Specifically, through the three steps of word segmentation, part-of-speech tagging, and named entity recognition (NER), the clean text is disassembled into "computable units" to provide structured input for subsequent three-dimensional entropy calculation. Each step needs to balance "universality" and "domain adaptability" to avoid precision loss of general tools in professional scenarios.

[0075] For example, first, word segmentation is performed, in which the core of word segmentation is to split continuous text into words (Chinese) or words (English) according to "semantic integrity", ensuring the accuracy of the granularity of subsequent part-of-speech tagging and NER. This embodiment segments according to "sentence boundaries" (period, question mark, exclamation mark) combined with "semantic topics" (such as "1. symptom description 2. diagnosis result"), with each segment length controlled at 50-100 characters. At the same time, each segment is independently segmented, and the associated structure of "paragraph ID-segmentation list" is output, which facilitates the segmented calculation of semantic entropy and sample entropy and the positioning of hallucinations (such as a segment containing "alcohol treatment of high blood pressure", which can be accurately marked as high risk).

[0076] Further, part-of-speech tagging is performed to tag each word in the segmentation result with grammatical attributes (such as noun, verb, adjective, conjunction). The core value of part-of-speech tagging is to "screen key semantic components" to filter meaningless grammatical units and improve subsequent computing efficiency. Semantic entropy calculation needs to analyze the "logical consistency" of the text (such as cause and effect, transition, parallelism), and logical relationships are mainly carried by "conjunctions" (such as "because, therefore, but, at the same time"). Therefore, the conjunctions (part-of-speech tag "c") are highlighted in this embodiment, and a "conjunction position-conjunction type" list is output (such as "because → causal conjunction, but → transition conjunction"). At the same time, the conjunctions in long texts are counted, and if the conjunction density of a certain text is too high (such as "because... therefore... but... however..."), it is marked as "high logical complexity" in advance, and the logical coherence is checked during subsequent semantic entropy calculation.

[0077] Further, named entity recognition (NER) is performed to extract core verification objects. Specifically, "entities with specific meanings" (such as names, place names, disease names, product names) are extracted from the text, and the entity type and boundary are labeled. The core is to provide "core verification units" for subsequent sample entropy calculation (entity authenticity verification) and semantic entropy calculation (entity-attribute relationship construction), and the essence of illusion recognition is to "verify the authenticity of entities and their relationships". Without named entity recognition, sample entropy cannot locate the key information to be verified, and semantic entropy calculation needs to construct "entity-attribute-relation" triples (such as "nifedipine tablets - indication - hypertension"). Therefore, this embodiment further associates the attributes of the entity: based on "part-of-speech tagging + dependency syntax analysis", the attributes of the entity are extracted (such as "nifedipine tablets 5mg" → entity "nifedipine tablets", attribute "dose: 5mg"). Further output the "entity-attribute-relation" structured list as the input of semantic entropy calculation.

[0078] Finally, the standardized intermediate data is output. After text cleaning, segmentation, part-of-speech tagging, and NER, the dispersed processing results need to be integrated into a "standardized intermediate data format" to ensure that the subsequent core calculation layer (semantic entropy, available entropy, sample entropy engine) can be directly called, avoiding efficiency loss due to incompatible data formats.

[0079] Step 2: Based on the pre-processing results, the illusion situation is comprehensively evaluated through three dimensions of fact, logic and scene. Based on the structured data output in the pre-processing stage (including segmentation, part-of-speech tagging, NER and entity-attribute relationship information), the output of the large model is accurately quantified from the three core dimensions of "fact authenticity", "logical integrity" and "scene adaptability", and then the multi-dimensional evaluation results are analyzed by the "fusion decision module". Finally, the reliability of the large model output is determined, and the precise positioning of the illusion type is realized.

[0080] Among them, the fact truth judgment module is for "fact error type hallucination" (such as medical drug indication error, education knowledge point deviation, and financial interest rate value error), through "fact assertion splitting + external knowledge verification", the consistency degree of large model output and true fact is quantitatively evaluated, and the quantitative evaluation logic and index are as follows:

[0081] a. Fact assertion splitting, based on the "entity-attribute-relation" triplets in the preprocessed data, the large model output is split into the smallest "verifiable fact assertion unit", each assertion needs to include "core entity + attribute + attribute value" three elements. For example, for the original output: "hypertensive patients take 5mg nifedipine tablets daily, which can relieve headache", the split assertions are nifedipine tablet-indication-hypertension; nifedipine tablet-adult daily dose-5mg; nifedipine tablet- efficacy-relieve headache.

[0082] b. Double-source fact verification and quantitative scoring, build "authority knowledge graph + domain fact rule library" double verification source, evaluate the authenticity of each assertion, and output "fact consistency score (FCS)".

[0083] Specifically, the domain authority knowledge graph is called to calculate the matching degree of the assertion and the "entity-attribute-relation" in the graph, and the specific calculation formula is: matching degree=entity matching score×0.4+attribute matching score×0.3+attribute value matching score×0.3. Load the field-specific fact rules to check the compliance of the assertion, and output the rule compliance score (0-1). Then calculate the fact consistency score, FCS=(knowledge graph matching degree×0.6+rule compliance score×0.4) / total number of assertions.

[0084] In addition, for the evaluation accuracy of different application scenarios, the embodiment adopts for real-time response scene, only split "high-risk assertions" (such as drug dosage in medicine and interest rate value in finance), call knowledge graph cache data for quick matching, FCS calculation delay≤50ms, threshold is set to 0.6; for offline response scene, split all assertions, call knowledge graph full data, combined with dynamic update rule library, "attribute value fuzzy assertion" is added "value interval verification", and the score accuracy is improved.

[0085] Further, the logic integrity judgment module quantitatively evaluates the logic integrity and consistency degree of the large model output by constructing a "hierarchical logic graph", and the quantitative evaluation logic and index are as follows:

[0086] a. Construct a two-level logical graph based on the "logical conjunction annotation" and "entity-attribute relationship" in the preprocessed data. Among them, the sentence-level logic is to annotate the "subject-predicate-object logic" and "modification relationship logic" within a single sentence; the paragraph-level logic is to annotate the "cause-effect / transition / timing logic" across sentences.

[0087] b. Logic conflict detection and quantitative scoring, design "logic conflict detector", detect contradictions from "local logic" and "global logic" two dimensions, output "logic coherence score (LCS, Logic Coherence Score)": among them, the local logic conflict detection is to check the rationality of "entity-attribute" within a single sentence, and output the local conflict rate; the global logic conflict detection is to check the consistency of the cross-sentence logical relationship, and output the global conflict frequency. Then calculate the logic coherence score, the specific calculation formula is: LCS=1-(local conflict rate×0.5+global conflict frequency×0.1).

[0088] In addition, for the evaluation accuracy of different application scenarios, the embodiment adopts a lightweight logic rule engine for real-time response scenarios, only detects "sentence-level local logic", LCS calculation delay ≤30ms, threshold is set to 0.7; for offline response scenarios, detect "sentence-paragraph" two-level logic, introduce pre-trained logic inference model to assist in judging complex cause-effect / timing conflicts, and increase exclusive logic verification rules for education "historical event timing", medical "diagnosis process" and other scenes.

[0089] Further, the scene adaptability judgment module quantitatively evaluates the adaptability of the large model output to the original problem scene through "scene feature extraction + theme matching" for "irrelevant content hallucination", and the quantitative evaluation logic and indicators are as follows:

[0090] a. Based on the "scene identification" in the preprocessed data, construct a domain-specific "scene feature library", each scene contains a set of core entities, a set of theme keywords, and a set of irrelevant features.

[0091] b. Based on the preprocessed text segmentation and entity information, calculate the "scenario relevance score (SRS, Scenario Relevance Score)". Among them, the scenario relevance score is obtained by integrating the core entity matching rate, the keyword coverage rate and the content proportion, the core entity matching rate is the proportion of the scene "core entity set" contained in the large model output; the keyword coverage rate is the cosine similarity between the output text and the scene "theme keyword set"; the content proportion is the proportion of the number of words containing "irrelevant feature blacklist" in the output; then calculate the relevance score, the specific calculation formula is: SRS=core entity matching rate×0.4+theme keyword coverage rate×0.4-irrelevant content proportion×0.2.

[0092] In addition, for the evaluation accuracy of different application scenarios, the embodiment adopts "core entity matching rate" and "irrelevant content proportion" for real-time response scenarios, uses FAISS fast retrieval core entity, and SRS calculation delay is less than or equal to 40 ms. The threshold is set to 0.65 (SRS < 0.65 is marked as "high irrelevant content risk"). For offline response scenarios, "theme keyword depth matching" is added, the theme similarity of the calculated output and the sample is calculated in combination with the scene sample library, and "theme deviation penalty" is added for financial "compliance documents", educational "textbook chapters" and other scenarios.

[0093] Step 3: Through the four steps of "standardization-weight distribution-comprehensive calculation-threshold judgment", the multi-dimensional results are coordinated, and finally the "comprehensive credibility score" and "hallucination type positioning" are output. The specific steps are as follows:

[0094] First step: quantization standardization. Since FCS, LCS and SRS are scores in the interval [0, 1], but the evaluation dimensions are different (FCS focuses on facts, LCS focuses on logic, and SRS focuses on scenes), the "Z-score standardization" is used to eliminate the influence of score distribution difference in different scenes, and ensure the comparability of scores in each dimension:

[0095] ;

[0096] Among them, is the original score (FCS / LCS / SRS), is the mean value of the score in the corresponding scene historical labeled data, is the standard deviation; the standardized score is mapped to the interval [0, 1], and the formula is:

[0097] ;

[0098] Among them, is the final result (through the Sigmoid function, the score is concentrated in the interval 0.2-0.8, to avoid the interference of extreme values).

[0099] Second step: field adaptive weight distribution. According to the priority requirements of "facts, logic and scenes" in different application scenarios, dynamic weights are allocated to the standardized FCS, LCS and SRS (FCS, LCS, SRS, and , , , and ), the weight can be adjusted through "field expert labeling + reinforcement learning optimization", and the core scene weight is as follows:

[0100] Third step: comprehensive credibility score calculation. Specifically, the "comprehensive credibility score (CCS)" output by the large model is calculated by "weighted summation + risk penalty", and the formula is as follows:

[0101] ;

[0102] Wherein, is the initial evaluation result of fact evaluation, is the initial evaluation result of logic evaluation, is the initial evaluation result of scene evaluation, is the high-risk dimension mark, is the preset risk penalty coefficient, is the weight of FCS, is the weight of LCS, is the weight of SRS, and .

[0103] Fourth step: dynamic threshold judgment and illusion positioning. The specific content is as follows:

[0104] a. Dynamic threshold setting. Based on the "historical annotation data set" of the corresponding scene, the ROC curve analysis method is used to determine the "illusion judgment threshold (T)", to ensure that the accuracy is ≥ 90% under the premise of minimizing the missed rate. For example, for the medical scene, the illusion judgment threshold is determined as T=0.6; for the e-commerce scene, the illusion judgment threshold is determined as T=0.55; for the education / finance scene, the illusion judgment threshold is determined as T=0.58.

[0105] b. Precise positioning of illusion type. Combined with the deviation degree of each dimension score and threshold, the specific illusion type is positioned. For example, if FCS<scene FCS threshold and |FCS-threshold| is maximum: it is determined as "fact error type illusion", and the high-risk fact assertion is output; if LCS<scene LCS threshold and |LCS-threshold| is maximum: it is determined as "logic contradiction type illusion", and the conflicting logic relationship is output; if SRS<scene SRS threshold and |SRS-threshold| is maximum: it is determined as "irrelevant content type illusion", and the irrelevant feature is output; if multiple dimensions are lower than the threshold: it is determined as "composite type illusion", and the main illusion type is output according to the deviation degree.

[0106] In some embodiments, the target entity, the target entity attribute of the target entity, and the association relationship of the target entity are extracted from the initial reply in step S120, including the following steps S210 to S230:

[0107] Step S210, text disassembly is performed on the initial reply to obtain a plurality of text segments;

[0108] In step S220, word segmentation, part-of-speech tagging, and named entity recognition are performed on each text segment to obtain the word segmentation result of each text segment, the part-of-speech tagging information of each text segment, and the target entity of each text segment.

[0109] In step S230, based on the word segmentation result of each text segment and the part-of-speech tagging information of each text segment, the attribute content corresponding to the target entity and the association relationship between the target entities are analyzed to obtain the target entity attribute and the entity association relationship.

[0110] In this embodiment, the initial reply is textually disassembled to obtain multiple text segments, thereby avoiding cross-segment semantic interference. Then, word segmentation, part-of-speech tagging, and named entity recognition are performed on each text segment to obtain the word segmentation result of each text segment, the part-of-speech tagging information of each text segment, and the target entity.

[0111] The text disassembly adopts a punctuation-based or semantic segmentation algorithm to divide the text segments, ensuring semantic coherence. The word segmentation is implemented by using a natural language processing tool, for example, a bidirectional long short-term memory network-based word segmentation model. The part-of-speech tagging is performed by using a pre-trained language model to classify the part-of-speech of the word segmentation result. The named entity recognition is performed by using a conditional random field or deep learning-based entity recognition model to extract the target entity.

[0112] Further, based on the word segmentation result of each text segment and the part-of-speech tagging information of each text segment, the attribute content corresponding to the target entity and the association relationship between the target entities are analyzed. For example, the entity attribute is extracted by using dependency syntax analysis or semantic role labeling, and the attribute content can be determined according to the modification relationship between adjectives and nouns. The association relationship between the entities is constructed by using co-occurrence analysis or a graph neural network.

[0113] Thus, the embodiment realizes the conversion from unstructured text to structured information through multiple steps such as text disassembly, word segmentation, and part-of-speech tagging, improves the accuracy and completeness of information extraction, and can effectively extract structured entity, attribute, and relationship information from the initial reply, thereby providing a basis for subsequent multidimensional hallucination evaluation.

[0114] In some embodiments, before the initial reply is subjected to multidimensional hallucination evaluation based on the target entity, the target entity attribute of the target entity, and the association relationship of the target entity in step S130 to obtain the initial evaluation result corresponding to each dimension in the multidimensional hallucination evaluation, the following step S310 is included:

[0115] In step S310, the evaluation accuracy of the multidimensional hallucination evaluation is determined according to the application scenario of the question and answer request. The application scenario of the question and answer request includes one of the following: a real-time response scenario and an offline response scenario, and the evaluation accuracy of the offline response scenario is greater than that of the real-time response scenario.

[0116] In the embodiment, the evaluation accuracy of the multi-dimensional hallucination evaluation is determined according to the application scenario of the question and answer request. The application scenario of the question and answer request includes a real-time response scenario and an offline response scenario, and the evaluation accuracy of the offline response scenario is greater than that of the real-time response scenario.

[0117] Specifically, the determination of the application scenario is achieved by analyzing the context semantics and interaction delay requirements of the question and answer request. The real-time response scenario corresponds to an interactive task that requires millisecond-level feedback in the dialogue system, and the offline response scenario corresponds to a non-immediate task that allows asynchronous processing. In addition, the adjustment of the evaluation accuracy can be achieved by dynamically configuring the knowledge graph retrieval depth, the logical rule verification level, and the scene feature matching granularity. For example, the real-time scenario adopts entity fuzzy matching and core logic chain detection, and the offline scenario enables entity accurate matching and full logic network traversal. The evaluation accuracy parameter is quantified as a knowledge graph node expansion level threshold, the real-time scenario is set to two-level associated node retrieval, and the offline scenario is expanded to five-level associated node retrieval.

[0118] Therefore, the embodiment can flexibly adjust the evaluation accuracy according to the specific application scenario and field, and balance between response speed and accuracy. For the real-time scenario with high time sensitivity, a lower evaluation accuracy is used to ensure response speed; for the offline scenario with higher accuracy requirements, a higher evaluation accuracy is used to improve recognition effect, so as to adapt to the needs of different scenarios and improve the practicality and adaptability of the system.

[0119] In some embodiments, in step S130, the initial reply is subjected to multi-dimensional hallucination evaluation based on the target entity, the target entity attribute of the target entity, and the association relationship of the target entity, to obtain an initial evaluation result corresponding to each dimension in the multi-dimension, including the following steps S410 to S450:

[0120] Step S410, based on the target entity, the target entity attribute, and the entity association relationship, the initial reply is disassembled into a plurality of fact units; the fact unit represents the factual content of the initial reply, and the fact unit includes a fact entity, a fact attribute, and a fact attribute value;

[0121] Step S420, according to the application scenario of the initial reply, a knowledge graph corresponding to the initial reply and a fact rule set corresponding to the initial reply are called;

[0122] Step S430, the matching degree of each fact unit and the knowledge graph corresponding to the initial reply is calculated;

[0123] Step S440, according to the fact rule set corresponding to the question and answer request, each fact unit is subjected to rule verification to obtain a corresponding verification score;

[0124] Step S450, according to the matching degree of each fact unit and the verification score of each fact unit, the initial evaluation result corresponding to the fact evaluation is calculated.

[0125] In the embodiment, the initial reply is disassembled to form a plurality of fact units, each unit containing entities, attributes and attribute values, wherein the disassembly of the initial reply is preferably to decompose the initial reply into verifiable fact units containing entities, attributes and attribute values by text parsing technology, for example, using dependency syntax analysis to extract entities and attributes in the subject-predicate-object structure.

[0126] Further, according to the application scenario of the initial reply, the knowledge graph corresponding to the initial reply and the fact rule set corresponding to the initial reply are called. For example, the medical knowledge graph is called in the medical scenario, and the commodity knowledge base is called in the e-commerce scenario. Further, the matching degree of each fact unit and the knowledge graph corresponding to the initial reply is calculated, preferably using vector similarity calculation method, the entities in the fact unit and the knowledge graph are vectorized, and then the cosine similarity between the vectors is calculated as the matching degree.

[0127] Further, according to the fact rule set corresponding to the question and answer request, the rule verification is performed on each fact unit to obtain the corresponding verification score. Among them, the rule verification preferably verifies the rationality of the attribute value through the pre-defined logical expression, for example, verifying whether the numerical attribute value meets the threshold range or the enumeration attribute is in the allowed set.

[0128] Further, according to the matching degree of each fact unit and the verification score of each fact unit, the initial evaluation result corresponding to the fact evaluation is calculated. Preferably, the matching degree and the verification score are combined to obtain the final fact evaluation result by using weighted average method.

[0129] Therefore, the embodiment can identify potential factual errors in more fine-grained manner by disassembling the initial reply into fact units and combining the knowledge graph and the fact rule set for evaluation. Moreover, by calculating the matching degree and performing rule verification, the fact can be verified from multiple angles, reducing the possibility of misjudgment, so as to perform multi-dimensional fact evaluation on the initial reply, improve the accuracy and comprehensiveness of illusion identification, and further not only suitable for single field, but also can adapt to different application scenarios by calling different knowledge graphs and rule sets, improving the flexibility and adaptability of illusion identification.

[0130] In some embodiments, in step S130, the initial reply is subjected to multi-dimensional illusion evaluation based on the target entity, the target entity attribute of the target entity, and the association relationship of the target entity, to obtain the initial evaluation result corresponding to each dimension in the multi-dimensional evaluation, further comprising the following steps S510 to S530:

[0131] Step S510, constructing a hierarchical logical network of the initial reply based on the target entity attribute and the entity association relationship; the hierarchical logical network includes a sentence-level logical chain and a paragraph-level logical chain, the sentence-level logical chain represents a mapping relationship between the target entity and the target entity attribute in each text segment in the initial reply, and the paragraph-level logical chain represents a connection logic of the target entity association relationship between the text segments in the initial reply.

[0132] Step S520, calling a preset logical rule library to perform local logical detection and global logical detection on the hierarchical logical network respectively to obtain a local detection result and a global detection result; the local logical detection is used for detecting the sentence-level logical chain; and the global logical detection is used for detecting the paragraph-level logical chain.

[0133] Step S530, calculating an initial evaluation result corresponding to the logic evaluation according to the local detection result and the global detection result.

[0134] In this embodiment, a hierarchical logical network corresponding to the initial reply is constructed based on the target entity attribute and the entity association relationship, and the hierarchical logical network includes a sentence-level logical chain and a paragraph-level logical chain. The sentence-level logical chain represents a mapping relationship between the target entity and the target entity attribute in each text segment in the initial reply, and the paragraph-level logical chain represents a connection logic of the target entity association relationship between the text segments in the initial reply.

[0135] Preferably, when the hierarchical logical network is constructed, the sentence-level logical chain generates an entity attribute mapping graph by analyzing the subject-predicate-object structure in a single text segment, for example, in a medical consultation scene, mapping “patient body temperature 39℃” as a numerical relationship between the entity “patient” and the attribute “body temperature”. The paragraph-level logical chain generates a logical connection path by analyzing the transition words, time adverbs and pronouns of adjacent text segments, for example, in an e-commerce consultation scene, establishing a cause-and-effect relationship between “goods out of stock” and “expected to be restocked in three days” through the time adverb.

[0136] Further, the preset logical rule library is called to perform local logical detection and global logical detection on the hierarchical logical network respectively to obtain a local detection result and a global detection result. The preset logical rule library includes a sentence-level rule set and a paragraph-level rule set, the sentence-level rule set includes an attribute value type checking rule and a contradictory attribute detection rule, and the paragraph-level rule set includes a logical coherence rule and a cause-and-effect rationality rule.

[0137] Specifically, the local logical detection is used for detecting the sentence-level logical chain, and specifically checks whether the relationship between the entity and the attribute in each sentence-level logical chain conforms to the preset rule; and the global logical detection is used for detecting the paragraph-level logical chain, and specifically checks whether the entity association in the paragraph-level logical chain is coherent and consistent.

[0138] Further, it is preferred to integrate the local detection result and the global detection result by using a weighted average method or the like to calculate the initial evaluation result corresponding to the logical evaluation. Thus, the present embodiment can evaluate the internal logic of a sentence and the logic between paragraphs by constructing a hierarchical logical network, thereby improving the accuracy and comprehensiveness of the logical evaluation and realizing multi-level and comprehensive logical evaluation on the initial reply. Meanwhile, the logical detection in the local and global dimensions can complement each other, which can effectively identify the logical contradictions and inconsistencies in the reply, helping to more accurately identify the logical illusion problem in the model generated content and improving the reliability of the illusion identification.

[0139] In some embodiments, the initial reply is subjected to multi-dimensional illusion evaluation based on the target entity, the target entity attribute of the target entity, and the association relationship of the target entity in step S130 to obtain the initial evaluation result corresponding to each dimension in the multi-dimensional evaluation. The method further includes the following steps S610 to S630:

[0140] In step S610, the scene feature library corresponding to the application scenario of the question and answer request is called, and the scene feature library includes a scene entity set, a scene keyword set, and a scene irrelevant identifier.

[0141] In step S620, the entity matching rate, the keyword coverage rate, and the irrelevant content proportion of the initial reply are calculated according to the scene feature library.

[0142] In step S630, the entity matching rate, the keyword coverage rate, and the irrelevant content proportion are integrated to obtain the initial evaluation result corresponding to the scene evaluation.

[0143] In the present embodiment, the scene feature library corresponding to the application scenario of the question and answer request is called, which has good scene adaptability and can meet the differentiated needs of illusion identification in different fields such as medical treatment and finance. The scene feature library includes a scene entity set, a scene keyword set, and a scene irrelevant identifier. Preferably, the construction of the scene feature library is realized by pre-setting the core entity set and the field keyword set of different application scenarios, and the scene irrelevant identifier includes entities and words that are common in each scene but irrelevant to the current scene.

[0144] For example, for the medical diagnosis scene, the scene entity set includes medical related entities such as disease name, symptoms, and drugs; the scene keyword set includes medical field commonly used words such as “diagnosis”, “treatment”, and “medication”; and the scene irrelevant identifier includes words such as “weather” and “stock” that are irrelevant to medical treatment.

[0145] Further, according to the scene feature library, the entity matching rate, the keyword coverage rate and the irrelevant content proportion of the initial reply are calculated. Specifically, the entity matching rate is calculated by counting the intersection proportion of the target entity and the scene entity set, the keyword coverage rate is calculated by the word frequency statistics and the matching degree of the scene keyword set, and the irrelevant content proportion is determined by detecting the proportion of the scene irrelevant identifier contained in the initial reply.

[0146] Further, the initial evaluation results corresponding to the scene evaluation are obtained by integrating the entity matching rate, the keyword coverage rate and the irrelevant content proportion in a weighted average manner, wherein the weights of the entity matching rate, the keyword coverage rate and the irrelevant content proportion can be adjusted according to specific application scenarios.

[0147] Therefore, the embodiment can perform multi-dimensional scene relevance evaluation on the initial reply based on the scene feature library, quantify the matching degree of the initial reply and the specific scene from the aspects of entity, keyword and irrelevant content, improve the accuracy and comprehensiveness of scene evaluation, effectively identify the content irrelevant to the application scene or deviating from the theme, and thus improve the accuracy of hallucination identification.

[0148] In some embodiments, in step S140, the hallucination identification result of the target model is generated according to the initial evaluation results, including steps S710 to S740 as follows:

[0149] In step S710, the initial evaluation results of the fact evaluation, the initial evaluation results of the logic evaluation and the initial evaluation results of the scene evaluation are integrated to obtain the target evaluation result.

[0150] In step S720, the corresponding hallucination threshold is called according to the application scene of the question and answer request.

[0151] In step S730, the target evaluation result is compared with the hallucination threshold to obtain a comparison result.

[0152] In step S740, if the target evaluation result is higher than or equal to the hallucination threshold, the hallucination identification result without hallucination is generated; if the target evaluation result is lower than the hallucination threshold, the initial identification result with hallucination is generated, and the hallucination type of the initial reply is located according to the initial evaluation results; the hallucination type is determined based on the initial evaluation results and the corresponding hallucination evaluation threshold; the initial identification result and the hallucination type of the initial reply are combined to obtain the hallucination identification result.

[0153] In this embodiment, the initial evaluation results of the factual evaluation, the initial evaluation results of the logical evaluation and the initial evaluation results of the scene evaluation are preferably integrated by a weighted sum formula to obtain a target evaluation result. Then, according to the application scenario of the question and answer request, a corresponding illusion threshold is called. For example, in the medical consultation scenario, the illusion threshold is set to 0.9 to ensure high accuracy; in the e-commerce customer service scenario, the illusion threshold is set to 0.8 to balance efficiency and accuracy.

[0154] Further, the target evaluation result is compared with the illusion threshold to obtain a comparison result. Preferably, the comparison result is determined by calculating the difference between the target evaluation result and the illusion threshold. If the target evaluation result is higher than or equal to the illusion threshold, an illusion-free illusion recognition result is generated; if the target evaluation result is lower than the illusion threshold, an initial recognition result with illusion is generated, and the illusion type of the initial reply is located according to the illusion evaluation threshold of the corresponding dimension according to each initial evaluation result, and then the initial recognition result and the illusion type of the initial reply are combined to obtain the illusion recognition result.

[0155] Illustratively, in the medical consultation scenario, if the target evaluation result is 0.95, which is higher than the illusion threshold 0.9, an illusion-free illusion recognition result is generated. In the e-commerce customer service scenario, if the target evaluation result is 0.75, which is lower than the illusion threshold 0.8, the evaluation results of each dimension are further analyzed; if the factual evaluation result is 0.7, which is lower than the factual evaluation threshold 0.8, it is determined that there is factual illusion, and then an illusion recognition result of "there is factual illusion" is generated.

[0156] Thus, the present embodiment realizes multi-dimensional illusion evaluation on the initial reply, generates accurate illusion recognition results according to the evaluation results, improves the accuracy and interpretability of illusion recognition, and can be flexibly adjusted according to different application scenarios, effectively reducing the negative impact of illusion on actual application. At the same time, by locating the specific illusion type, a clear direction is provided for subsequent reply optimization, thereby improving the reliability and practicality of large language models in actual application.

[0157] As shown in Figure 2 some embodiments of the present application provide a multi-dimensional recognition model illusion system, which includes an acquisition module 210, an extraction module 220, an evaluation module 230, a result module 240, specifically:

[0158] The acquisition module 210 is configured to acquire the initial reply of the target model in response to the target model generating the corresponding initial reply in response to the question and answer request.

[0159] The extraction module 220 is used to extract the target entity, the target entity attributes, and the relationship between the target entity from the initial response; the target entity is extracted after the text is decomposed into multiple text fragments based on the initial response;

[0160] Evaluation module 230 is used to perform multi-dimensional illusion evaluation on the initial response based on the target entity, the target entity attributes, and the relationship between the target entities, to obtain the initial evaluation result corresponding to each dimension. The multi-dimensional illusion evaluation includes factual evaluation, logical evaluation, and scenario evaluation. Factual evaluation is a quantitative evaluation based on factual content associated with the target entity. Logical evaluation is a quantitative evaluation based on logical rules associated with the target entity attributes and entity relationship. Scenario evaluation is a quantitative evaluation based on scenario relevance associated with the target entity and the target entity attributes.

[0161] Results module 240 is used to generate illusion recognition results for the target model based on the initial evaluation results.

[0162] It should be noted that the system for identifying multidimensional model illusions provided in this embodiment is based on the same inventive concept as the method for identifying multidimensional model illusions described above. Therefore, the relevant content of the method for identifying multidimensional model illusions described above also applies to the content of the system for identifying multidimensional model illusions. Therefore, it will not be repeated here.

[0163] To this end, the system responds to the target model's response to the question-and-answer request by generating a corresponding initial response, thus obtaining the target model's initial response; it extracts the target entity, the target entity's target entity attributes, and the target entity's relationships from the initial response; based on the target entity, the target entity's target entity attributes, and the target entity's relationships, it performs a multi-dimensional illusion evaluation of the initial response to obtain the initial evaluation result for each dimension; the multi-dimensional illusion evaluation includes factual evaluation, logical evaluation, and scenario evaluation; based on each initial evaluation result, it generates the target model's illusion recognition result. In this way, it can achieve multi-dimensional evaluation through entity extraction, and then combine the multi-dimensional evaluation results to identify illusions, realizing a multi-dimensional evaluation of the factual authenticity, logical completeness, and scenario adaptability of the model's output content, and accurately identifying and locating the illusion type.

[0164] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-mentioned method for multi-dimensional recognition model illusion.

[0165] like Figure 3 , Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes:

[0166] at least one battery;

[0167] at least one memory;

[0168] at least one processor;

[0169] at least one program;

[0170] The program is stored in the memory, and the processor executes the at least one program to implement the method of the multi-dimensional identification model illusion provided in the present disclosure.

[0171] The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, and the like.

[0172] The electronic device of the embodiment of the present application is described in detail below.

[0173] The processor 1600 can be implemented in a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute related programs to implement the technical solutions provided by the embodiments of the present disclosure.

[0174] The memory 1700 can be implemented in the form of a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present disclosure are implemented by software or firmware, the related program codes are stored in the memory 1700 and are called and executed by the processor 1600 to implement the method of the multi-dimensional identification model illusion provided in the present disclosure.

[0175] The input / output interface 1800 is used to realize information input and output.

[0176] The communication interface 1900 is used to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0177] The bus 2000 transmits information between various components (for example, the processor 1600, the memory 1700, the input / output interface 1800, and the communication interface 1900) of the device.

[0178] The processor 1600, the memory 1700, the input / output interface 1800, and the communication interface 1900 are communicatively connected with each other within the device through the bus 2000.

[0179] The disclosure also provides a storage medium, which is a computer-readable storage medium, and stores computer executable instructions for causing a computer to execute the method for multi-dimensional identification model hallucination.

[0180] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0181] The embodiments described in the disclosure are used to more clearly illustrate the technical solutions of the disclosure, and do not constitute a limitation on the technical solutions provided by the disclosure. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the disclosure are also applicable to similar technical problems.

[0182] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the disclosure, and can include more or fewer steps than the figures, or combine certain steps, or different steps.

[0183] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separated, that is, they can be located in one place, or distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiments.

[0184] Those skilled in the art can understand that all or some steps in the above disclosed method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware, and appropriate combinations thereof.

[0185] The terms "first", "second", "third", "fourth", and the like in the description of this application and in the claims hereof, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed herein is solely for the convenience of the reader and does not limit the scope of the application. It is also to be understood that the description and examples in this application are intended to cover all possible combinations where any of the several elements can represent one or more elements.

[0186] It should be understood that, in this application, "at least one" means one or more, "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c, can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0187] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0188] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment of the present application.

[0189] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0190] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0191] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above implementation. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the embodiments of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the embodiments of the present application.

[0192] The embodiments of the present application are described in detail above in combination with the drawings, but the present application is not limited to the above embodiments. Those skilled in the art can make various changes within the scope of knowledge possessed by them without departing from the purpose of the present application.

Claims

1. A method for multi-dimensional identification of model hallucinations, characterized in that, The method includes: In response to the target model's response to the question-and-answer request, generate a corresponding initial response and obtain the target model's initial response; The target entity, its attributes, and its relationships are extracted from the initial response; the target entity is obtained by breaking down the initial response into multiple text fragments. Based on the target entity, its target entity attributes, and its associations, the initial response undergoes a multi-dimensional illusion assessment to obtain an initial assessment result for each dimension. This multi-dimensional illusion assessment includes factual assessment, logical assessment, and scenario assessment. The factual assessment is a quantitative evaluation based on factual content associated with the target entity. The logical assessment is a quantitative evaluation based on logical rules associated with the target entity attributes and its associations. The scenario assessment is a quantitative evaluation based on scenario relevance associated with the target entity and its attributes. Based on the initial evaluation results, the hallucination recognition results of the target model are generated.

2. The method for multi-dimensional identification of hallucinations according to claim 1, characterized in that, Extracting the target entity, its attributes, and its relationships from the initial response includes: The initial response was broken down into multiple text fragments; Each of the text segments is segmented, part-of-speech tagging is performed, and named entity recognition is performed to obtain the segmentation results, part-of-speech tagging information, and target entities of each text segment. Based on the word segmentation results and part-of-speech tagging information of each text fragment, the attribute content corresponding to the target entity and the association between each target entity are analyzed to obtain the target entity attributes and the entity association.

3. The method for multi-dimensional identification of hallucinations according to claim 1, characterized in that, Before performing a multi-dimensional illusion assessment on the initial response based on the target entity, the target entity attributes of the target entity, and the association relationships of the target entity to obtain the initial assessment result corresponding to each of the multi-dimensional dimensions, the method further includes: determining the assessment accuracy of the multi-dimensional illusion assessment according to the application scenario of the question-and-answer request; the application scenario of the question-and-answer request includes one of the following: real-time response scenario and offline response scenario, and the assessment accuracy of the offline response scenario is greater than the assessment accuracy of the real-time response scenario.

4. The method for multi-dimensional identification of hallucinations according to claim 1, characterized in that, The initial response is evaluated using a multi-dimensional illusion assessment based on the target entity, its target entity attributes, and the relationships between the target entities, to obtain an initial assessment result for each of the multiple dimensions, including: Based on the target entity, the target entity attributes, and entity relationships, the initial response is decomposed into multiple fact units; each fact unit represents the factual content of the initial response, and each fact unit includes a fact entity, a fact attribute, and a fact attribute value. Based on the application scenario of the initial response, the knowledge graph corresponding to the initial response and the fact rule set corresponding to the initial response are invoked; Calculate the matching degree between each of the fact units and the knowledge graph corresponding to the initial response; Based on the fact rule set corresponding to the question and answer request, rule verification is performed on each fact unit to obtain the corresponding verification score; Based on the matching degree of each fact unit and the verification score of each fact unit, the initial evaluation result corresponding to the fact evaluation is calculated.

5. The method for multi-dimensional identification of hallucinations according to claim 1, characterized in that, The step of performing a multi-dimensional illusion assessment on the initial response based on the target entity, the target entity attributes of the target entity, and the association relationships of the target entity, to obtain the initial assessment result corresponding to each of the multi-dimensional aspects, further includes: Based on the target entity attributes and entity associations, a hierarchical logic network for the initial response is constructed. The hierarchical logic network includes sentence-level logic chains and paragraph-level logic chains. The sentence-level logic chains represent the mapping relationship between target entities and target entity attributes within each text segment of the initial response, and the paragraph-level logic chains represent the connection logic of target entity associations between each text segment of the initial response. A preset logic rule library is invoked to perform local logic detection and global logic detection on the hierarchical logic network, respectively, to obtain local detection results and global detection results; the local logic detection is used to detect the sentence-level logic chain; the global logic detection is used to detect the paragraph-level logic chain; Based on the local detection results and the global detection results, calculate the initial evaluation result corresponding to the logical evaluation.

6. The method for multi-dimensional identification of hallucinations according to claim 1, characterized in that, The step of performing a multi-dimensional illusion assessment on the initial response based on the target entity, the target entity attributes of the target entity, and the association relationships of the target entity, to obtain the initial assessment result corresponding to each of the multi-dimensional aspects, further includes: Based on the application scenario of the question-and-answer request, the corresponding scenario feature library is retrieved. The scenario feature library includes a scenario entity set, a scenario keyword set, and scenario-independent identifiers. Based on the scenario feature library, calculate the entity matching rate, keyword coverage rate, and proportion of irrelevant content in the initial response; By integrating the entity matching rate, the keyword coverage rate, and the proportion of irrelevant content, the initial evaluation result corresponding to the scenario evaluation is obtained.

7. The method for multi-dimensional identification of hallucinations according to claim 1, characterized in that, The step of generating the hallucination recognition result of the target model based on each of the initial evaluation results includes: By integrating the initial evaluation results of the factual assessment, the initial evaluation results of the logical assessment, and the initial evaluation results of the scenario assessment, the target evaluation result is obtained. Based on the application scenario of the question-and-answer request, the corresponding hallucination threshold is invoked; The target evaluation result is compared with the hallucination threshold to obtain the comparison result; If the target evaluation result is higher than or equal to the hallucination threshold, a hallucination recognition result without hallucination is generated; if the target evaluation result is lower than the hallucination threshold, an initial recognition result with hallucination is generated, and the hallucination type of the initial response is located according to each initial evaluation result; the hallucination type is determined based on each initial evaluation result and the hallucination evaluation threshold of the corresponding dimension; the hallucination recognition result is obtained by combining the initial recognition result and the hallucination type of the initial response.

8. A system for multi-dimensional identification of model hallucinations, characterized in that, The system includes: The acquisition module is used to acquire the initial response of the target model in response to the question-and-answer request and generate the corresponding initial response. The extraction module is used to extract the target entity, the target entity attributes, and the association relationships of the target entity from the initial response; the target entity is obtained by extracting the target entity after decomposing the text into multiple text fragments based on the initial response; An evaluation module is used to perform a multi-dimensional illusion evaluation on the initial response based on the target entity, the target entity attributes, and the association relationships of the target entity, to obtain an initial evaluation result corresponding to each of the multi-dimensional illusion evaluations. The multi-dimensional illusion evaluation includes factual evaluation, logical evaluation, and scenario evaluation. The factual evaluation is a quantitative evaluation based on factual content associated with the target entity. The logical evaluation is a quantitative evaluation based on logical rules associated with the target entity attributes and the entity association relationships. The scenario evaluation is a quantitative evaluation based on scenario relevance associated with the target entity and the target entity attributes. The results module is used to generate the illusion recognition results of the target model based on the initial evaluation results.

9. An electronic device, characterized in that, The device includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor to enable the at least one control processor to perform a method for recognizing multidimensional model illusions as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a method for identifying multidimensional model illusions as described in any one of claims 1 to 7.

Images