Methods and systems for reducing hallucinations during interaction with artificial intelligence

Abstract

The present invention relates to techniques for reducing and evaluating hallucinations in a generative artificial intelligence (gAI), comprising the steps of: a) Receiving an input (prompt) containing an instruction to the gKI, b) Configuring an input to the gAI based on predefined output specifications, wherein the output specifications influence the gAI in generating the response to reduce hallucinations, c) Generating an initial response by the AI ​​based on the configured output specifications, d) Calculating an uncertainty assessment of the first response using the gCI, which describes the potential hallucination risk of the response, and e) Providing the first answer depending on the uncertainty assessment and / or its confidence interval.

Description

[0001] The present invention relates to the field of artificial intelligence (AI) and, in particular, to the reduction and / or evaluation of so-called hallucinations in generative artificial intelligences (gAIs). gAIs are systems that generate synthetic content based on user input, e.g., in the form of text or image data (prompts). Such systems are used in a variety of application areas, such as text processing, image and video analysis, automation technologies, and in safety-critical areas such as medical technology or autonomous driving. The invention focuses on methods and systems that contribute to increasing the accuracy and reliability of the outputs generated by gAIs and thus expanding their range of applications.

[0002] Generative artificial intelligence (gAI) has made significant progress in recent years and is increasingly being used in commercial, scientific, and safety-critical fields. These systems are typically based on large neural networks trained on extensive datasets. They generate content such as text, images, or other media based on user input. Despite their ability to understand complex relationships and generate content, gAIs are susceptible to so-called "hallucinations." These are situations in which the AI ​​system generates information that is factually incorrect or fabricated, but often appears plausible. Such hallucinations pose a significant risk to users who rely on the correctness and accuracy of the AI ​​output.

[0003] The phenomenon of hallucinations arises from several factors. First, the way neural networks recognize data patterns and make probability-based predictions can lead the AI ​​to generate seemingly plausible but incorrect information. Second, inconsistent or insufficient training data leads to inconsistencies in the responses of the gAI. Third, contradictory instructions in the prompt or poorly defined inputs are also possible causes that lead the AI ​​to rely on speculative statements.

[0004] Current technology offers only limited options for preventing hallucinations in gAIs. Some conventional methods aim to improve the training dataset or increase the amount of data. However, these measures are often insufficient to completely prevent hallucinations, especially in situations where user input is complex or ambiguous. A precise and automatic assessment of uncertainty in gAI responses, as well as targeted control of output parameters prior to response generation, represents a gap in current technology. Therefore, there is a need for a method that reduces hallucinations and increases the reliability of gAI outputs.

[0005] The object of the present invention is to provide a technical method that reduces the susceptibility of generative artificial intelligences (gAIs) to hallucinations and improves the accuracy and reliability of the content generated by the gAI. In particular, the method should enable a precise assessment of the uncertainty in the generated responses and promote the generation of more reliable results by adjusting the parameters in the response process. By introducing mechanisms for variably adjusting the output parameters and for calculating an uncertainty assessment, the invention can significantly reduce the probability of false or speculative responses (hallucinations).

[0006] The present invention solves this problem through its independent claims.

[0007] According to the invention, a method for reducing and evaluating hallucinations in a generative artificial intelligence (gAI) is specified, comprising the steps, in particular the steps are carried out according to the following sequence: a) Receiving an input (prompt) containing an instruction to the gKI, b) Configuring an input to the AI ​​based on predefined output specifications, whereby the output specifications influence the AI ​​in generating the response; if the output specifications are already contained as information in the prompt, for example, the output specifications can be transmitted via the prompt, in which case the input would correspond to the prompt; if the output specifications are in a configuration file, the prompt can be modified before being passed to the AI ​​so that a new prompt is generated which contains the output specifications; c) Generating an initial response by the AI ​​based on the configured output specifications, d) Calculating an uncertainty assessment of the first response using the gCI, which describes the potential hallucination risk of the response, and e) Providing the first answer depending on the uncertainty assessment and / or its confidence interval.

[0008] This describes the basic procedure for reducing hallucinations, which begins with the input (prompt) and is controlled by specific output parameters. The configuration influences the AI ​​in such a way that the generated response is based on certain criteria, such as simplicity or fact-basedness. Uncertainty assessment helps to identify potential hallucinations. The response is delivered based on this assessment. This method makes it possible to significantly reduce the probability of hallucinations during response generation, thus increasing the reliability of the AI ​​outputs.

[0009] The calculation of the uncertainty assessment is described in detail in German patent application 10 2024 116 857, the disclosure of which is hereby fully incorporated into the application.

[0010] In summary, the calculation of the uncertainty assessment can be described as follows: Uncertainty assessment is used to calculate the intrinsic accuracy of the responses generated by a generative AI (gAI) to queries. This involves calculating not only the accuracy score but also a confidence interval that indicates the range of possible uncertainties. This process enables the AI ​​to better evaluate its responses and ensure that inconsistent or potentially hallucinatory content is clearly identified and, if necessary, avoided. In this way, the AI ​​can deliver more reliable results and filter out potentially inaccurate information. Steps for calculating the uncertainty assessment: a) Processing the request and assigning it to subject areas

[0011] First, the generative AI receives a request (prompt) that is assigned to a specific topic area. The AI ​​uses a mapping logic to identify the topic area based on the request content. This is crucial because the calculation of the uncertainty assessment and the selection of the most reliable languages ​​depend on the AI ​​knowing the thematic framework in which the request belongs. b) Assessment of the correctness value

[0012] After processing the prompt and assigning the topic, the AI ​​generates its answer. A key step here is the calculation of a so-called accuracy score. This score is a numerical expression that represents the internal accuracy of an answer on a scale of 0 to 100. • Scale from 0 to 100: ◯ An accuracy score of 100 means that the AI ​​is extremely confident in its answer being correct. ◯ Values ​​close to 0 indicate very low reliability; the AI ​​considers the answer to be potentially inaccurate or unsuitable for the context.

[0013] The accuracy score is the result of the AI's internal probability calculations, based on the availability and consistency of the data in the training set that can be used to answer the respective query. The scale from 0 to 100 is flexible enough in this respect to reflect minor differences in the assessment and to transparently represent larger variations in the data, depending on the context. c) Calculation of a confidence interval for uncertainty assessment

[0014] In addition to the accuracy score, a confidence interval is calculated to indicate the range of response reliability. The confidence interval provides two additional values: a lower and an upper limit, which indicate with a certain probability (e.g., 98%) the range within which the true accuracy score is expected to lie. • Calculation of the confidence interval: ◯ The AI ​​determines the width of the interval based on the dispersion of responses it can obtain from the training data for the topic and query language. A narrow interval, such as [98, 100], indicates high data consistency and therefore a more reliable answer. A wide interval, such as [72, 92], indicates greater uncertainty in the data.

[0015] The confidence interval provides additional information on response accuracy: A narrower interval suggests a more consistent and stable data situation, while a wide interval indicates an increased dispersion of the training data and thus a potential risk of hallucination. Application of uncertainty assessment to multiple languages ​​and subject areas

[0016] The AI ​​can also apply the uncertainty assessment process to multiple languages ​​to improve the data situation and identify the most reliable answer: • The request will first be translated into languages ​​spoken by people with strong skills in the relevant subject area. • Each answer in the different languages ​​is then assigned an individual accuracy score and confidence interval, enabling the AI ​​to identify the best language versions. • By comparing the answers, the AI ​​can then select the answer with the highest accuracy value and the narrowest confidence interval and display it to the user as the most reliable answer.

[0017] This procedure ensures that the AI ​​not only provides an answer in the input language, but also delivers the most accurate answer overall, even if it may need to be translated back from another language.

[0018] Advantages and technical impact of uncertainty assessment • Improved response quality: By linking the accuracy score and the confidence interval, AI can specifically reduce uncertainties and hallucinations. The answers become more reliable and precise, which is particularly advantageous for safety-critical applications. • Automatic selection of the most reliable language: The AI ​​can specifically choose the language for a query whose training data is best suited to the specific topic. This significantly improves data quality and therefore the response quality. • Transparency for the user: Displaying the accuracy score and confidence interval gives the user a better understanding of the answer quality and the reliability of the AI's assessment. This allows users to make more informed decisions, such as requesting alternative answers or adjusting the prompt. • Application to different queries and subject areas: The uncertainty assessment methodology can be universally applied to different subject areas and query types, as the AI ​​calculates the internal accuracy values ​​individually and thus adapts to the specific context of the query.

[0019] In one embodiment, the output specifications are configured to define the response complexity, length, and / or fact-based nature of the response.

[0020] This specifies the configuration parameters that control the response: response complexity, length, and / or fact-basedness. These parameters can be flexibly adjusted to achieve a simpler, shorter, and / or fact-based response that is less prone to hallucinations. Parameter control allows for targeted adaptation to user requirements and offers the advantage of increasing the precision of the gAI output without intervention.

[0021] In one embodiment, the uncertainty assessment and confidence interval are only output if the uncertainty assessment exceeds a predetermined threshold indicating an increased risk of hallucinations.

[0022] This controls the conditional output of the uncertainty rating and confidence interval. Only if the uncertainty rating exceeds a threshold is it displayed as an additional warning of potential hallucinations. This mechanism relieves the user of unnecessary information and focuses attention only on uncertain results. This improves usability and ensures that the user receives only relevant warnings.

[0023] In one embodiment of the method, the uncertainty assessment in step d) is specified on a scale of 1 to 1000 and, in addition, a 99% confidence interval is calculated to determine the reliability of the uncertainty assessment.

[0024] This specifies the scale for the uncertainty assessment and the confidence interval. A scale from 1 to 1000 allows for a fine-grained evaluation of the response reliability, and the 99% confidence interval provides a precise estimate of the result accuracy. This methodology helps the user better assess the reliability of the response, which is particularly beneficial in safety-critical applications.

[0025] In one embodiment, the method further comprises the following steps: f) Automatically vary the output specifications of the AI ​​to generate multiple variants of the first answer, with the output specifications being changed in predefined steps, and / or g) Conducting a cross-comparison of the different response variants to identify the response with the lowest risk of hallucination.

[0026] These features extend the procedure to include the generation of multiple response variants and cross-comparison. The output parameters are modified in predefined steps to generate different response variants, which are then compared. This approach allows the version with the lowest risk of hallucination to be identified and preferentially displayed. This reduces the probability of an erroneous response and improves consistency.

[0027] In one embodiment of the method according to, the variation of the output specifications in step f) includes an adjustment of the response complexity, response length and / or fact-basedness at predetermined intervals, wherein each combination of the output specifications leads to the generation of a separate response variant.

[0028] The parameters of the variation are thus refined, particularly the complexity and fact-based nature of the responses. By adjusting these parameters at intervals, different response variants can be generated, based on a broad spectrum of possible outcomes. These combinations increase the probability of finding the most precise and consistent answer possible.

[0029] In one embodiment, the method further comprises the step: h) generating a recommendation for the user based on the uncertainty assessment and cross-comparison of the different answer options, wherein the recommendation indicates to the user the most reliable answer option.

[0030] This adds a recommendation component that guides the user to the most reliable answer option. The recommendation is based on the uncertainty assessment and a comparison of the answer options. This step provides the user with guidance and simplifies the decision for the optimal answer. It increases usability and minimizes the user's effort in checking the answer quality.

[0031] In one embodiment of the method, if the predetermined threshold of the uncertainty assessment is exceeded, the gKI generates a suggestion for improving the input (prompt) in order to minimize potential hallucinations.

[0032] This introduces an additional safety measure that optimizes the prompt in cases of high uncertainty. If the uncertainty assessment exceeds the threshold, a suggestion for improving the input is generated, which can mitigate potential hallucinations in advance. This mechanism represents a further way to increase response accuracy and reduce the risk of hallucinations.

[0033] In one embodiment, the method improves the accuracy and reliability of the answers generated by the gAI by reducing the risk of hallucination, thereby increasing the consistency and trustworthiness of the output for the user.

[0034] This describes another technical effect of the method: the increase in the accuracy and reliability of AI responses through hallucination reduction. This effect improves the consistency and trustworthiness of the outputs and increases the usability of gAI in areas where high precision is required.

[0035] According to another aspect of the invention, a system for reducing and evaluating hallucinations in a generative artificial intelligence (gAI) is proposed, comprising: a) an input device, in particular a microphone and / or a keyboard, for receiving a prompt, b) a configuration unit that defines or applies output specifications for the gAI, which influence the gAI when generating the response; the configuration unit can be designed as a processor capable of handling configuration data and / or prompts; the configuration unit defines the input that is passed to the gAI; if the output specifications are already contained in the prompt as information, for example, the output specifications can be transmitted via the prompt, in which case the input would correspond to the prompt and the configuration unit merely forwards the prompt; if the output specifications are contained in a configuration file, the prompt can be modified before being passed to the gAI so that a new prompt is created that contains the output specifications; c) a processing unit, in particular a processor on which the gAI is implemented and for processing the input, for generating an initial response based on the configured output specifications, d) an evaluation unit, in particular a processor with a suitable algorithm or evaluation AI, for calculating an uncertainty assessment of the first response describing the potential hallucination risk, and e) an output device, in particular a screen and / or a loudspeaker, to provide the initial response, wherein the uncertainty rating and confidence interval are only output if the uncertainty rating exceeds a threshold.

[0036] The system is specifically designed to execute the procedure. It includes units for input, configuration, processing, and evaluation. The evaluation unit ensures that the uncertainty assessment is only displayed in cases of high risk. The system optimizes efficiency and provides the technical infrastructure necessary for implementing the procedure.

[0037] Preferred embodiments of the present invention are explained below with reference to the accompanying figure: Fig. Figure 1 shows the system and method according to the invention.

[0038] Numerous features of the present invention are explained in detail below with reference to preferred embodiments. The present disclosure is not limited to the specific combinations of features mentioned. Rather, the features mentioned here can be combined arbitrarily to form embodiments according to the invention, unless expressly excluded below.

[0039] Fig. Figure 1 schematically shows the system 100 for calculating the uncertainty assessment of a generative artificial intelligence (gAI) and also the method according to the invention. The system comprises several functional units that are arranged in a linear process flow and communicate with each other. 1. Input Unit 110 and Step 110a: Input Unit 110 receives the input request or prompt from the user. This unit ensures that the request is correctly and completely transmitted to the system to enable precise processing in the following units. 2. Configuration Unit 120, Step 120a: Configuration Unit 120 defines specific parameters for the gAI. These parameters, also known as output specifications, can include characteristics such as response complexity, length, and fact-basedness. They serve to control AI generation in a targeted manner, ensuring that the generated responses optimally meet the requirements of the query. 3. Processing Unit 130 and Step 130a: Processing Unit 130 handles the actual generation of the response based on the configured output specifications. Here, the request is interpreted by the AI ​​and the corresponding response is generated, which is then evaluated in subsequent steps regarding its accuracy and uncertainty. 4. Assessment Unit 140 and Step 140a: Assessment Unit 140 calculates an uncertainty score for the generated answer. This score includes a correctness value on a scale and a confidence interval that assesses the reliability of the answer and takes into account possible variations or hallucinations. 5. Output Unit 150 and Step 150a: Output Unit 150 provides the final answer to the user. Depending on the uncertainty rating, the unit may output additional information such as the correctness value and the confidence interval, particularly if the uncertainty rating exceeds a certain threshold.

[0040] Units 110 to 150 are arranged so that the data is passed from one unit to the next, ensuring efficient and orderly processing of the request and evaluation of the response.

[0041] The technical problem solved by the invention is to develop a method that induces genetically engineered artificial intelligence (GI) systems to reduce the probability and severity of hallucinations in their outputs. This includes, for example, identifying specific technical parameters (e.g., response complexity, length, and fact-basedness) that directly affect the accuracy of the responses, as well as introducing a method for assessing response uncertainty. This task requires inventive technical considerations because it is an inherent problem of GI systems, which generate their responses based on statistical patterns but lack "understanding" or verification mechanisms.

[0042] The invention is based on the understanding that gAIs can be specifically influenced by controlling the response parameters in their output processes, thereby reducing the tendency to hallucinations. This control not only improves the quality of the generated responses but also requires a more advanced technical solution that introduces uncertainty assessment based on defined parameters (e.g., response length and fact-basedness). This method for reducing hallucinations goes beyond conventional methods for improving gAIs and includes automatic parameter control and response verification, representing a novel technical approach. By combining response parameter control with an uncertainty analysis unit, a technically precise and efficient solution is provided that overcomes existing limitations of the prior art.

[0043] A preferred embodiment of the invention is described below: The proposed solution for reducing hallucinations and calculating uncertainty assessments in generative artificial intelligence (gAI) comprises a series of steps and mechanisms that enable gAI to configure its responses in a targeted manner, independently assess uncertainties, and identify and improve potentially unsafe responses. This procedure ensures that gAI not only generates its responses but also systematically tests and evaluates them to guarantee high reliability and precision.

[0044] A key part of the solution involves providing the gAI with specific output specifications via a prompt or configuration file. These specifications define how the response should be generated with respect to various parameters that influence the likelihood of hallucinations. Key parameters include the complexity, length, and fact-basedness of the response. A lower specification for response complexity means the AI ​​considers fewer aspects, reducing the risk of hallucinations because the AI ​​is less inclined to include uncertain or speculative information. The simpler the response, the lower the risk of the AI ​​presenting speculative content as certain. Conversely, a high specification for the fact-basedness of the response can also reduce the likelihood of hallucinations, as the AI ​​focuses more on verified information.However, this can limit the creativity of the answers, as the AI ​​has less scope to develop alternative or creative, but potentially speculative, solutions.

[0045] These parameters can be defined specifically in the prompt, and the input can be provided as a predefined value in the format of a numerical scale that specifies the desired level of each aspect. An example of such a prompt is: "Dear AI, please generate an answer to the task that satisfies the following aspects on a scale of 1 to 10, where 10 represents the highest level of expression:" • Simplicity of answer: 5 • Length of answer: 3 • Fact-basedness of the answer: 9”

[0046] This prompt gives the AI ​​precise instructions regarding the desired response quality, ensuring that the answer meets the user's expectations in terms of clarity, length, and reliability. Configuring the output specifications thus provides an effective way to influence the quality and consistency of the response and significantly reduce the risk of hallucinations.

[0047] Another key mechanism of the solution is the automatic calculation of an uncertainty score for each generated answer. This uncertainty score is based on a correctness value, which indicates the internal accuracy of the answer on a scale of 1 to 1000. A value of 1000 signals that the AI ​​is extremely confident its answer is correct, while lower values ​​indicate greater uncertainty and thus an increased risk of hallucinations. In addition to the correctness value, the AI ​​calculates a confidence interval that reflects the range of uncertainties. A narrow interval—for example, [997, 1000]—indicates high consistency and strong reliability, while a wide interval, such as [900, 950], suggests variations in the data and therefore a less certain answer.

[0048] The uncertainty rating and confidence interval can be output by the system on request at the prompt or via the configuration file. An example of a specific input for the uncertainty rating could be: "Dear AI, please rate your output regarding its uncertainty that a hallucination may be at least partially present. Provide a rating on a scale of 1 to 1000 (with 1000 being the maximum confidence that no hallucination is present) and include the 99% confidence interval. Example: Hallucination rating 998 [997, 1000]."

[0049] This uncertainty assessment allows the AI ​​to provide the user with a transparent picture of its response quality. A low accuracy score and a wide confidence interval signal uncertainties and enable the user to critically examine the answer or adjust the prompt accordingly.

[0050] In addition to assessing uncertainty, the AI ​​can automatically generate multiple answer variations by selectively varying predefined output parameters. These variations can be defined in the prompt or the configuration file. For example, the AI ​​can systematically vary the answer simplicity, such as by setting values ​​like 5, 7, and 9. Alternatively, the configuration can specify that the answer simplicity varies from a baseline value within a defined range. For instance, if the simplicity is set to 7 and a variation of ±2 is specified, the AI ​​will automatically generate answer simplicity values ​​of 5 and 9.

[0051] Based on these variations, the AI ​​generates several answer options, each of which is individually evaluated. A cross-comparison of these options allows the AI ​​to identify inconsistencies between the answers and to pinpoint the option with the lowest risk of hallucination. The selected answer option represents the most reliable and consistent response and can be presented to the user as the preferred solution.

[0052] Should the uncertainty rating or confidence interval exceed certain thresholds, the system can provide the user with a recommendation for selecting the best answer option. This recommendation is based on an objective assessment of the uncertainties and helps the user choose the answer that best suits their needs. Even if no thresholds are exceeded, the AI ​​can optionally display the rating scores to provide the user with transparency regarding the assessment.

[0053] If the uncertainty assessment indicates a high risk of hallucinations, the AI ​​can also analyze the prompt or configuration to identify the source of the uncertainty. Should the uncertainty be caused by unclear or contradictory instructions in the prompt, the AI ​​can automatically improve the prompt or provide the user with targeted suggestions for adjustments. These suggestions can be iterative, so the user receives a new response after each adjustment. This process is repeated until a satisfactory and reliable response is achieved.

[0054] The insights gained from uncertainty assessments and cross-comparisons of response variants can be used to optimize the AI ​​system in the long term. The evaluation data can be used to improve the quality of the training data, adjust specific parameters of response generation, or optimize plugins and agents. This continuously increases the system's long-term performance.

[0055] Overall, the proposed solution offers a comprehensive methodology for uncertainty assessment and hallucination reduction in generative AI. The process is fully automatable and ideally runs invisibly to the user. Only when certain thresholds are exceeded does the user receive additional assistance for evaluating or adjusting the response, which significantly increases the transparency and reliability of the AI ​​output. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2024 116 857

[0009]

Claims

A method for reducing and evaluating hallucinations in a generative artificial intelligence (gAI), comprising the steps of: a) receiving an input containing an instruction to the gAI, b) configuring an input to the gAI based on predefined output specifications, where the output specifications influence the gAI in generating the response for hallucination reduction, c) generating an initial response by the gAI based on the configured output specifications, d) calculating an uncertainty rating of the initial response by the gAI, describing the potential hallucination risk of the response, and e) providing the initial response depending on the uncertainty rating and / or its confidence interval. Method according to claim 1, wherein the output specifications define the response complexity, length and / or the fact-based nature of the response. Method according to claim 1 or 2, wherein the uncertainty assessment and the confidence interval are only output if the uncertainty assessment exceeds a predetermined threshold indicating an increased risk of hallucination. Method according to one of claims 1 to 3, wherein the uncertainty assessment in step d) is specified on a scale of e.g. 1 to 1000 and additionally a 99% confidence interval is calculated to determine the reliability of the uncertainty assessment. A method according to any one of claims 1 to 4, further comprising the steps: f) automatically varying the output specifications of the gAI to generate multiple variants of the first response, wherein the output specifications are changed in predetermined steps, and / or g) performing a cross-comparison of the different response variants to identify the response with the lowest risk of hallucination. Method according to claim 5, wherein the variation of the output specifications in step f) comprises an adjustment of the response complexity, the response length and / or the fact-basedness at predetermined intervals, wherein each combination of the output specifications leads to the generation of a separate response variant. Method according to one of claims 5 or 6, further comprising the step: h) generating a recommendation for the user based on the uncertainty assessment and cross-comparison of the different answer options, wherein the recommendation indicates to the user the most reliable answer option. Method according to any one of claims 1 to 7, wherein, if the predetermined threshold of the uncertainty assessment is exceeded, the gKI generates a suggestion for improving the input (prompt) in order to minimize potential hallucinations. Method according to any one of claims 1 to 8, wherein reducing the risk of hallucination improves the accuracy and reliability of the answers generated by the gAI, thereby increasing the consistency and trustworthiness of the output for the user. A system for reducing and evaluating hallucinations in a generative artificial intelligence (gAI), comprising: a) an input unit for receiving a prompt, b) a configuration unit that takes into account output specifications for the gAI when generating an input to the gAI, which influence the gAI in generating the response, c) a gAI as a processing unit for generating an initial response based on the configured output specifications, d) an evaluation unit for calculating an uncertainty rating of the initial response that describes the potential risk of hallucination, and e) an output unit for providing the initial response, wherein the uncertainty rating and the confidence interval are only output if the uncertainty rating exceeds a threshold.

Images