Model hallucination suppression method, apparatus, device, storage medium, and program product

Abstract

The application discloses a model hallucination suppression method and device, equipment, a storage medium and a program product, relates to the technical field of large models, and the application obtains a current confidence state signal of a large model before a content generation flow of the large model generates a target content unit, realizes real-time monitoring of a content generation process, adjusts a control parameter related to the content generation flow according to the current confidence state signal and a preset ideal confidence state, that is, advances an intervention opportunity from after the fact to before the fact, realizes active control of hallucination generation conditions, guides the generation behavior of the target content unit based on the adjusted control parameter, and maintains the confidence state signal of the large model at the preset ideal confidence state, thereby realizing suppression of hallucination of the large model. The technical problem that the after-verification and correction mode is difficult to fundamentally suppress the generation of model hallucination is overcome.

Description

Technical Field

[0001] This application relates to the field of large model technology, and in particular to methods, apparatus, devices, storage media and program products for suppressing model illusions. Background Technology

[0002] Large models often contain factual errors, logical contradictions, or unfounded fabrications in the generated content, i.e., model illusions exist, which seriously hinder their widespread application in high-reliability scenarios.

[0003] Currently, hallucination suppression methods typically involve introducing an external knowledge base to post-process the model's generated results for verification and correction. However, with this approach, errors have already been generated, and post-processing cannot completely eliminate their potential impact, making it difficult to fundamentally suppress the generation of model hallucinations. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, device, storage medium, and program product for suppressing model hallucinations, aiming to solve the technical problem that post-verification and correction methods are insufficient to fundamentally suppress the generation of model hallucinations.

[0005] To achieve the above objectives, this application proposes a method for suppressing model hallucinations, the method comprising: Before the target content unit is generated from the content generation stream of the large model, the current confidence state signal of the large model is obtained; Based on the current confidence state signal and the preset ideal confidence state, adjust the control parameters related to the content generation stream; Based on the adjusted control parameters, the generation behavior of the target content unit is guided, so that the confidence state signal of the large model is maintained at the preset ideal confidence state, thereby suppressing the illusion of the large model.

[0006] In one embodiment, the content generation stream includes a text generation stream, the control parameters include decoding parameters, and the step of adjusting the control parameters related to the content generation stream based on the current confidence state signal and a preset ideal confidence state includes: Based on the current confidence state signal, determine the current confidence level of the large model; Based on the current confidence level and the preset confidence interval, a first adjustment mode is determined, wherein the preset confidence interval is used to define the preset ideal confidence state; According to the first adjustment mode, the decoding parameters related to the text generation stream are adjusted.

[0007] In one embodiment, the first adjustment mode includes a focusing mode and a reflection mode, and the step of determining the first adjustment mode based on the current confidence level and a preset confidence interval includes: When the current confidence level is lower than the lower limit of the preset confidence level interval, the first adjustment mode is determined to be the focus mode; When the current confidence level is higher than the upper limit of the preset confidence interval, the first adjustment mode is determined to be the reflection mode.

[0008] In one embodiment, the step of adjusting the decoding parameters associated with the text generation stream according to the first adjustment mode includes: When the first adjustment mode is the focus mode, the decoding parameters related to the text generation stream are adjusted based on a preset convergence intervention strategy. The preset convergence intervention strategy is used to suppress low-probability words. When the first adjustment mode is the reflection mode, the decoding parameters related to the text generation stream are adjusted based on a preset divergent intervention strategy. The preset divergent intervention strategy is used to inject uncertainty to guide the model to self-correct.

[0009] In one embodiment, the decoding parameters include at least one of a temperature parameter, a k value of Top-k, and a p value of Top-p; The step of adjusting the decoding parameters related to the text generation stream based on a preset convergence intervention strategy includes at least one of the following: The temperature parameter is lowered by a preset first value, wherein the temperature parameter is used to control the smoothness of the output probability distribution; Lower the k value in Top-k by a preset second value; The p value in Top-p is lowered by a preset third value.

[0010] In one embodiment, the step of adjusting the decoding parameters related to the text generation stream based on a preset divergent intervention strategy includes: The temperature parameter is increased to a preset fourth value, and a preset self-questioning prompt is inserted into the content generation stream to participate in content generation.

[0011] In one embodiment, prior to the step of inserting preset self-questioning prompts into the content generation stream, the method further includes: The current generated content of the large model is obtained, and the preset self-questioning prompt words are generated based on the current generated content by the preset prompt optimization model. The preset prompt optimization model is obtained by supervised learning based on preset context content, different adjustment modes and different reflection prompts.

[0012] In one embodiment, the current confidence state signal includes at least one of output layer features and intermediate layer feature variances, wherein the output layer features include policy entropy and transition word features; the step of determining the current confidence of the large model based on the current confidence state signal includes at least one of the following: The policy entropy or intermediate layer feature variance is normalized to obtain a normalized value, which is then used as the current confidence level of the large model. The transition word features are standardized to obtain normalized transition word features. Based on preset weight coefficients, the policy entropy and the normalized transition word features are weighted and summed to obtain the current confidence of the large model.

[0013] In one embodiment, the step of determining the first adjustment mode based on the current confidence level and a preset confidence interval includes: Based on the current confidence level and the preset confidence interval, determine whether there is an anomaly in the current confidence state of the large model; If it exists, the lightweight verification model is invoked, and the external verification signal is output through the lightweight verification model, wherein the lightweight verification model is obtained by supervised training based on a historical fact dataset of a specific domain; The first adjustment mode is determined based on the external verification signal and the preset confidence interval.

[0014] In one embodiment, after the step of determining whether the current confidence state of the large model is abnormal based on the current confidence level and a preset confidence interval, the method further includes: If it exists, then record the hallucination-related data of the large model, wherein the hallucination-related data includes at least one of the current input context, current generated content and current confidence state signal of the large model; Based on the hallucination-related data, the large model is fine-tuned to generate content based on the fine-tuned large model.

[0015] In one embodiment, before the step of determining the first adjustment mode based on the current confidence level and the preset confidence interval, the method further includes: The generation performance of the large model is evaluated based on a preset set of verification questions, wherein the generation performance includes a trade-off between accuracy and abandonment rate. Based on the generated performance, the preset confidence interval is output through a confidence optimization algorithm, wherein the confidence optimization algorithm is used to search for a target confidence interval that optimizes the trade-off index.

[0016] In one embodiment, when the content generation stream is a text generation stream, the target content unit includes target lexical units, the current confidence state signal includes policy entropy and transition word features, and the step of obtaining the current confidence state signal of the large model includes: Obtain the output probability distribution of the previous word corresponding to the target word, and the output text currently generated by the large model; Calculate the policy entropy of the output probability distribution; The output text is scanned according to a preset transition word dictionary to determine the transition word features in the output text. The transition word features are used to reflect the inference state of the model.

[0017] In one embodiment, the step of adjusting the control parameters related to the content generation stream based on the current confidence state signal and a preset ideal confidence state includes: Based on the current confidence state signal, a second adjustment mode is output by the reinforcement learning agent. The reinforcement learning agent is trained by taking the historical confidence state signal as the state input, the adjustment mode as the action output, and the validity evaluation signal of the generated content as the basis. The validity evaluation signal of the generated content is used to calibrate the boundary of the preset ideal confidence state. Based on the second adjustment mode, the control parameters related to the content generation stream are adjusted.

[0018] Furthermore, to achieve the above objectives, this application also proposes a model hallucination suppression device, which includes: The acquisition module is used to acquire the current confidence state signal of the large model before the target content unit is generated from the content generation stream of the large model; The adjustment module is used to adjust the control parameters related to the content generation stream based on the current confidence state signal and the preset ideal confidence state. The guidance module is used to guide the generation behavior of the target content unit based on the adjusted control parameters, so that the confidence state signal of the large model is maintained at the preset ideal confidence state, thereby suppressing the illusion of the large model.

[0019] In addition, to achieve the above objectives, this application also proposes a model hallucination suppression device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the model hallucination suppression method as described above.

[0020] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the model illusion suppression method described above.

[0021] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the model illusion suppression method described above.

[0022] One or more technical solutions proposed in this application have at least the following technical effects: This application acquires the current confidence state signal of the large model before the target content unit is generated, enabling real-time monitoring of the content generation process. Based on the current confidence state signal and a preset ideal confidence state, control parameters related to the content generation flow are adjusted, i.e., the intervention timing is moved from post-hoc to pre-hoc. Simultaneously, the control parameters related to the content generation flow are adjusted according to different confidence state signals and the preset ideal confidence state, achieving proactive control over the conditions for hallucination generation. Thus, based on the adjusted control parameters, the generation behavior of the target content unit is guided. Since the control parameters related to the content generation flow are adjusted under the constraint of the preset ideal confidence state, the intervention occurs before the hallucination content is generated and directly affects the generation behavior of the target content unit. The method described in this application can maintain the confidence state signal of the large model at the preset ideal confidence state, thereby suppressing large model hallucinations. This overcomes the technical problem that post-hoc verification and correction methods are insufficient to fundamentally suppress the generation of model hallucinations. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating an embodiment of the hallucination suppression method for the model in this application. Figure 2 This is a first logical architecture diagram provided for Embodiment 2 of the hallucination suppression method of this application; Figure 3This is a second logical architecture diagram provided for Embodiment 2 of the hallucination suppression method of this application; Figure 4 This is a third logical architecture diagram provided for Embodiment 2 of the hallucination suppression method of this application; Figure 5 This is a schematic diagram of a scenario provided for Embodiment 2 of the hallucination suppression method of this application; Figure 6 This is a schematic diagram of the module structure of the hallucination suppression device in the embodiments of this application; Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the model illusion suppression method in this application embodiment.

[0026] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0027] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0028] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0029] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or a model hallucination suppression device capable of performing the above functions. The following description uses a model hallucination suppression device as an example to illustrate this embodiment and the subsequent embodiments.

[0030] Based on this, embodiments of this application provide a method for suppressing model hallucinations, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the hallucination suppression method of this application.

[0031] In this embodiment, the model hallucination suppression method includes steps S10 to S30: Step S10: Before the content generation stream of the large model generates the target content unit, obtain the current confidence state signal of the large model; Step S20: Adjust the control parameters related to the content generation stream based on the current confidence state signal and the preset ideal confidence state; Step S30: Based on the adjusted control parameters, guide the generation behavior of the target content unit so that the confidence state signal of the large model is maintained at the preset ideal confidence state, thereby suppressing the illusion of the large model.

[0032] It should be noted that large models often contain factual errors, logical contradictions, or unfounded fabricated information in the generated content, i.e., model illusion exists, which seriously hinders their widespread application in high-reliability scenarios. Currently, illusion suppression methods usually involve introducing external knowledge bases to post-process verify and correct the model's generated results. However, with this approach, the erroneous content has already been generated, and post-processing cannot completely eliminate its potential impact, making it difficult to fundamentally suppress model illusion.

[0033] In this embodiment, model illusions are proactively suppressed in advance during the model generation process, thereby avoiding the problem that post-processing cannot completely eliminate their potential impact.

[0034] Specifically, before the target content unit is generated by the content generation stream of the large model, model illusion is actively suppressed; where the large model can be a multimodal large model or a large language model, etc.

[0035] Among them, the content generation stream is used to output the content sequence step by step in time step or structural order during the model inference process. The content generation stream can be an image generation stream, a text generation stream, a video generation stream, a speech generation stream, etc.

[0036] It can be understood that the target content unit is the next basic generation unit that the large model will output during the inference process. The specific form of this basic generation unit depends on the type of content generation stream: in text generation streams, the basic generation unit is usually a token; in image generation streams, the basic generation unit is usually a pixel block, a latent space vector, or the output of a denoising step in the diffusion process; in speech generation streams, the basic generation unit is usually an audio sample point or an acoustic feature frame; and in video generation streams, the basic generation unit is usually a frame of image or a spatiotemporal block.

[0037] Before the target content unit is generated by the content generation stream of the large model, the model illusion suppression device, coupled with the internal inference interface of the large model, acquires the current confidence state signal of the large model in real time. This current confidence state signal is used to quantify the degree of certainty, consistency, or reliability of the model regarding the content to be generated under the current context.

[0038] Specifically, the current confidence state signals of large models corresponding to different content generation streams are different: in the autoregressive text generation of large language models, the model outputs a probability distribution of a token at each step, and the current confidence state signal includes at least the maximum probability value, the Shannon entropy of the output distribution, and the semantic consistency score with the context; in the diffusion model or autoregressive image generation model, the generation process is usually denoised or synthesized step by step in units of latent variables or pixel blocks, and the current confidence state signal includes at least the norm of the predicted noise residual of the current denoising step, and the discriminator's alignment score between the intermediate generation results and the text prompts.

[0039] When the content generation stream is a text generation stream, the target content unit includes target tokens, and the current confidence state signal includes policy entropy and transition word features. The implementation method for obtaining the current confidence state signal of the large model can be: Obtain the output probability distribution of the previous word corresponding to the target word, and the output text currently generated by the large model; calculate the policy entropy of the output probability distribution; scan the output text according to a preset transition word dictionary to determine the transition word features in the output text, the transition word features being used to reflect the inference state of the model.

[0040] Specifically, when the large model generates the t-th word (i.e., the target word), it obtains the output probability distribution of the previous word (the (t-1)-th word) corresponding to the target word, as well as the output text currently generated by the large model. The currently generated output text is the text up to step t. 1. The complete output text sequence generated by the model.

[0041] The output probability distribution can be expressed as:

[0042] Where v is the model vocabulary.

[0043] Calculate the Shannon entropy of the output probability distribution and use the Shannon entropy as the policy entropy;

[0044] A higher policy entropy value indicates a more dispersed and uncertain choice of the next term by the model, resulting in lower confidence. Conversely, a lower policy entropy value indicates that the model is highly concentrated on a few candidate terms, exhibiting stronger certainty and higher confidence. This policy entropy metric directly reflects the uncertainty of the model's internal reasoning and is a fundamental signal for assessing the risk of hallucination.

[0045] The output text is scanned according to a preset transition word dictionary to determine the transition word features in the output text. The preset transition word dictionary includes semantic function marker words, which are classified according to pragmatic categories as follows: transition words: such as, however, despite, although; uncertainty words: such as, possibly, perhaps, maybe; verification requirement words: such as, need to verify, let me check, in other words, etc.

[0046] This dictionary is used to perform keyword matching or regular expression-based scanning on the currently generated output text to statistically analyze transition word features, including the frequency of occurrence of transition words. freq Density of transition words density and the distance between the occurrence of transition words positionAt least one of the following, the transition word feature can be represented as: Boolean value, vector form ( Ft =[ freq , density , position ])wait.

[0047] It should be noted that the transition word features are used to indirectly reflect the model's current reasoning state and the strength of factual basis. For example, if the model frequently uses weak assertion words such as "maybe" and "it is said," it may indicate that it lacks reliable knowledge support and is in a low-confidence or fabrication-edge state. In this case, even if the policy entropy is low, potential illusions should still be guarded against.

[0048] By sensing and quantifying the inference state of the model in real time, it is beneficial to dynamically adjust the generation strategy before hallucinations occur, thereby achieving source-level hallucination suppression.

[0049] Furthermore, based on the current confidence state signal and the preset ideal confidence state, the control parameters related to the content generation stream are adjusted.

[0050] Among them, the preset ideal confidence state is a confidence state pre-set according to the reliability requirements of the target task. It is used to characterize the confidence level that the model should maintain under safe and reliable generation conditions. The preset ideal confidence state can be a confidence scalar or a multidimensional signal scalar.

[0051] The control parameters associated with the content generation stream differ depending on the content generation stream. When the content generation stream is a text generation stream, the control parameters include repetition penalties and retrieval enhancement switches. When the content generation stream is an image generation stream, the control parameters include diffusion step size noise scheduling, attention regularization strength, latent space cropping range, and classifier guidance scale.

[0052] Specifically, the implementation method of adjusting the control parameters related to the content generation stream based on the current confidence state signal and the preset ideal confidence state can be as follows: compare the current confidence state signal and the preset ideal confidence state to obtain the confidence deviation. Based on the sign and magnitude of the deviation, determine the adjustment direction and magnitude of the control parameters.

[0053] It should be noted that the implementation method for adjusting the control parameters related to the content generation stream based on the current confidence state signal and the preset ideal confidence state can be: Based on the current confidence state signal, a second adjustment mode is output by the reinforcement learning agent. The reinforcement learning agent is trained by using historical confidence state signals as state inputs, adjustment modes as action outputs, and a validity evaluation signal of the generated content. The validity evaluation signal of the generated content is used to calibrate the boundary of the preset ideal confidence state. Based on the second adjustment mode, control parameters related to the content generation stream are adjusted.

[0054] Specifically, the reinforcement learning agent is trained in an offline or online phase: specifically, the reinforcement learning agent is obtained by using historical confidence state signals as state inputs, adjustment modes as action outputs, and training based on the effectiveness evaluation signals of the generated content.

[0055] Among them, the State Space uses historical and current confidence state signals as input states, including but not limited to multi-dimensional signal sequences such as policy entropy, transition word features, attention weight variance, and activation value variance; the Action Space is defined as a set of adjustment modes, each mode corresponding to a set of control parameter configuration combinations; and the Reward Signal is constructed based on the effectiveness evaluation signal of the generated content, which is used to measure the comprehensive quality of the generated results in terms of factuality, logic, user satisfaction, or task completion.

[0056] The agent optimizes its policy by maximizing cumulative rewards, ultimately learning a policy function that selects the optimal adjustment mode given a sequence of confidence states. The validity evaluation signal of the generated content is used not only to train the agent but also to dynamically calibrate the boundaries of the preset ideal confidence state: if high rewards are frequently obtained within a certain confidence interval [0.65, 0.75] (i.e., the generated content is valid), the lower limit of the ideal confidence can be automatically lowered from 0.8 to 0.7; conversely, if hallucinations (low rewards) still occur at high confidence levels, an overconfidence warning mechanism may be triggered, introducing an upper limit constraint. Therefore, the preset ideal confidence state is no longer a static threshold but a dynamic target that can be continuously optimized through validity feedback.

[0057] During the inference phase, after obtaining the current confidence state signal, it is used as the state input to the trained reinforcement learning agent; the agent outputs the corresponding second adjustment mode; the control parameter configuration contained in the second adjustment mode is parsed and applied to the target content unit to be generated.

[0058] Furthermore, based on the adjusted control parameters, the generation behavior of the target content unit is guided, so that the confidence state signal of the large model is maintained at a preset ideal confidence state, thereby suppressing the illusion of a large model. That is, the adjusted control parameters are applied to the generation process in real time to actively shape the output distribution of the model, thereby constraining its behavioral trajectory at the source and causing its confidence state to converge towards the preset ideal interval.

[0059] Based on the adjusted control parameters described above, the following guiding effects can be achieved: When the original confidence level is too low, the output space is compressed, forcing the model to select content from high-probability, high-consistency regions to avoid fabrication. When the original confidence level is too high but there is a factual risk (e.g., a vague transition word is detected), appropriate diversity should be introduced or external validation should be triggered to prevent the solidification of errors.

[0060] In the multi-step generation process, a confidence feedback loop is formed. The result of each generation step affects the context of the next step, which in turn affects the new confidence state signal. Continuous fine-tuning keeps the model stable within the ideal confidence state. If the model's confidence state still cannot return to the ideal confidence state after multiple adjustments, a safety response strategy can be activated, such as: inserting a preset safety statement: "The accuracy of this information cannot be confirmed at present"; suspending generation and prompting the user for clarification or more information, etc.

[0061] This embodiment shifts the intervention timing from post-hoc correction to pre-hoc control. By dynamically adjusting control parameters during the generation process, it proactively guides the model's behavior towards an ideal confidence state, thereby suppressing the generation of illusions. Furthermore, it integrates the model's internal probability uncertainty (represented by policy entropy) with external linguistic behavioral cues (analyzing ambiguous, transitional, or confirmatory expressions in the generated text using a pre-defined transition word dictionary) to construct a multi-dimensional, highly sensitive real-time confidence state signal. This allows for earlier and more accurate perception of changes in the reliability of large models during the generation process. Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. On this basis, refer to Figure 2 , Figure 2 This is a first logical architecture diagram of the model hallucination suppression method in this embodiment. In this embodiment, the content generation stream includes a text generation stream, and the control parameters include decoding parameters. The implementation method for adjusting the control parameters related to the content generation stream according to the current confidence state signal and the preset ideal confidence state can be: Based on the current confidence state signal, determine the current confidence level of the large model; based on the current confidence level and a preset confidence interval, determine a first adjustment mode, wherein the preset confidence interval is used to define the preset ideal confidence state; based on the first adjustment mode, adjust the decoding parameters related to the text generation stream.

[0062] It should be noted that when the content generation stream is a text generation stream, the control parameters related to the content generation stream specifically include decoding parameters, such as repetition penalty coefficient and generation length limit, which are used to regulate the output behavior of the large language model in the autoregressive decoding process.

[0063] It should be noted that when the content generation stream is a text generation stream, the current confidence state signal includes at least one of the output layer features (policy entropy, transition word features) and intermediate layer features, wherein the variance of the attention weights and the variance of the activation values ​​are included.

[0064] Specifically, the step of determining the current confidence level of the large model based on the current confidence state signal includes at least one of the following: The policy entropy is normalized to obtain a normalized value, which is then used as the current confidence level of the large model. The transition word features are standardized to obtain normalized transition word features. Based on preset weight coefficients, the policy entropy and the normalized transition word features are weighted and summed to obtain the current confidence of the large model.

[0065] It is understandable that when processing the policy entropy, it can be directly normalized and mapped to a unified confidence metric space (e.g., the interval [0,1]) to obtain the corresponding normalized value; then, according to the semantic relationship (positive or negative correlation) between each signal and the confidence, the normalized value is directionally corrected, and finally the corrected value is used as the current confidence of the large model, or as a component of the current confidence.

[0066] When the transition word features include frequency, density, and position, the steps for standardizing the transition word features include: normalizing the density can be done by dividing by the maximum reasonable density value for linear normalization; normalizing the frequency can be done by using the maximum observation frequency in the training set or historical generated text; and normalizing the position can be achieved by mapping the position to a recency weight using an exponential decay function.

[0067] After the above processing, the sub-features are weighted and merged into a single normalized transition word feature component. The smaller the value, the stronger the uncertainty or inference fluctuation (i.e., the lower the confidence level).

[0068] Weight coefficients α and β are set to represent the degree of dependence on policy entropy and normalized transition word features, respectively. Based on the preset weight coefficients, the policy entropy and the normalized transition word features are weighted and summed to obtain the current confidence Ct of the large model.

[0069] Where, Ct=αsigmoid( Ht)+βnormalize(Ft).

[0070] Since policy entropy reflects the uncertainty of the probability distribution inside the model, and transition word features reflect the reasoning state expressed by the model through surface language behavior, the two characterize the confidence level from different dimensions: the model may still use words such as "it is said" and "maybe" under low entropy (high intrinsic confidence), implying that it is aware that the content lacks factual support; Conversely, training bias could lead to high entropy even when the actual content is correct. Therefore, the current confidence level obtained by weighting and summing the policy entropy and the normalized transition word features based on preset weight coefficients can more accurately determine the risk of hallucination.

[0071] Specifically, the step of determining the current confidence level of the large model based on the current confidence state signal may further include: determining the current confidence level of the large model based on intermediate layer features.

[0072] When the model is in a high-confidence state (such as deterministic reasoning based on sufficient knowledge), its attention mechanism tends to focus on key context areas, and the activation responses are highly consistent semantically, resulting in a concentrated distribution of attention weights and a small variance in activation values. When the model is in a low-confidence or illusion-edge state (such as lack of reliable evidence or forced completion), its internal representation tends to diverge, manifested as increased differences in activation responses and higher variance as attention is dispersed to multiple irrelevant locations, different channels, or spatial regions.

[0073] Therefore, the variance of intermediate layer features (the variance of attention weights and / or the variance of activation values) can be used to measure the intrinsic stability of the model.

[0074] The step of determining the current confidence level of the large model based on the current confidence state signal may further include: determining the current confidence level of the large model based on policy entropy and intermediate layer features, which is better at capturing early inference biases than relying solely on output layer entropy.

[0075] When the content generation stream is an image generation stream, the current confidence level can be defined as the joint uncertainty or internal representation variance of the generator-discriminator system. When the current confidence level is detected to be lower than the preset ideal range, perturbations or enhanced generation conditions can be actively applied to prompt the model to refocus on reliable semantic regions.

[0076] Further, a first adjustment mode is determined according to the current confidence and a preset confidence interval; the preset confidence interval is configured to define the preset ideal confidence state, and the boundary thereof can be pre-configured according to the safety requirements of an application scenario; the first adjustment mode corresponds to a set of specific decoding parameter configuration strategies.

[0077] The method for determining the first adjustment mode according to the current confidence and the preset confidence interval may be: matching a predefined first adjustment mode based on a relative position between the current confidence and the preset confidence interval; and adjusting decoding parameters related to the text generation flow according to the first adjustment mode.

[0078] Specifically, the first adjustment modes include a focus mode and a reflection mode, and the method for determining the first adjustment mode according to the current confidence and the preset confidence interval may further be: when the current confidence is lower than a lower limit of the preset confidence interval, determining the first adjustment mode as the focus mode; and when the current confidence is higher than an upper limit of the preset confidence interval, determining the first adjustment mode as the reflection mode.

[0079] It should be noted that the focus mode aims to compress the generation space and enhance deterministic guidance, and is suitable for low-confidence scenarios; the reflection mode aims to break solidified cognition and introduce external verification or diversity, and is suitable for overconfidence scenarios.

[0080] when the current confidence is lower than the lower limit of the preset confidence interval, determining the first adjustment mode as the focus mode; and when the current confidence is higher than the upper limit of the preset confidence interval, determining the first adjustment mode as the reflection mode.

[0081] Assuming that the preset confidence interval is [L, U], if Ct < L: it is determined that the model lacks confidence, and the focus mode is triggered; if Ct > U: it is determined that the model is overconfident, and the reflection mode is triggered; if L ≤ Ct ≤ U: the model is in an ideal state, and no intervention is performed.

[0082] Through the above dual-threshold-driven binary adjustment mechanism, the invention can not only inhibit fabrication under low confidence, but also prevent wrong conviction under high confidence, thereby realizing bidirectional active defense against hallucinations of large models.

[0083] Specifically, with reference to Figure 3 , Figure 3 which is a second logical architecture diagram of the model hallucination suppression method according to the embodiment, in this embodiment, the method for adjusting decoding parameters related to the text generation flow according to the first adjustment mode may be: When the first adjustment mode is the focus mode, the decoding parameters related to the text generation stream are adjusted based on a preset convergence intervention strategy, which is used to suppress low-probability words; when the first adjustment mode is the reflection mode, the decoding parameters related to the text generation stream are adjusted based on a preset divergence intervention strategy, which is used to inject uncertainty to guide the model to self-correct.

[0084] When the first adjustment mode is the focus mode, the decoding parameters related to the text generation stream are adjusted based on the preset convergence intervention strategy. The preset convergence intervention strategy aims to suppress the sampling possibility of low-probability, high-risk words and make the model output focus on the high-confidence, high-consistency region, thereby improving the factuality and reliability of the generated content.

[0085] When the first adjustment mode is the reflective mode, the decoding parameters related to the text generation stream are adjusted based on the preset divergent intervention strategy. The preset divergent intervention strategy aims to actively introduce controllable uncertainty or diversity, break the model's excessive solidification of erroneous knowledge, and prompt it to re-evaluate the current reasoning path and make self-corrections.

[0086] By employing the aforementioned two opposing but complementary intervention strategies of convergence and divergence, this application achieves bidirectional dynamic regulation of the generation behavior of large language models—it can tighten to prevent illusions at low confidence levels and promote self-reflection at high confidence levels, thereby significantly improving the credibility and robustness of the overall output while maintaining the fluency of generation.

[0087] Specifically, the decoding parameters include at least one of the following: temperature parameter, k value of Top-k, and p value of Top-p; the step of adjusting the decoding parameters related to the text generation stream based on a preset convergence intervention strategy includes at least one of the following: The temperature parameter is lowered by a preset first value, wherein the temperature parameter is used to control the smoothness of the output probability distribution; Lower the k value in Top-k by a preset second value; The p value in Top-p is lowered by a preset third value.

[0088] Accordingly, the step of adjusting the decoding parameters related to the text generation stream based on a preset convergence intervention strategy includes at least one of the following operations: The temperature parameter is lowered by a preset first value; wherein, the temperature parameter is used to control the smoothness of the model's output probability distribution: the higher the temperature value, the more uniform the distribution and the more random the generation; the lower the temperature value, the sharper the distribution, and high-probability words are significantly amplified. Lowering the temperature can effectively suppress the sampling of low-probability, high-risk words, causing the generation results to converge towards the high-confidence region.

[0089] The value of k in the Top-k sample is lowered by a preset second value; the Top-k sampling strategy samples only from the top k candidate lexical units with the highest probabilities. Lowering the value of k further restricts the generation space, eliminates low-quality options at the end, and enhances output determinism.

[0090] The P value in the Top-p algorithm is lowered by a preset third value. The Top-p (nucleus sampling) strategy dynamically selects the smallest set of terms whose cumulative probability first exceeds P as the sampling pool. Lowering the P value reduces this set, retaining only the most core high-probability terms, thereby improving the consistency and factual reliability of the generated content.

[0091] The aforementioned preset first, second, and third values ​​can be pre-set according to task sensitivity, model size, or user configuration. For example, in a high-reliability scenario, the temperature can be lowered from the default value of 1.0 to 0.4 (i.e., the first value is 0.6), the k value can be lowered from 50 to 20 (the second value is 30), and the p value can be lowered from 0.9 to 0.6 (the third value is 0.3).

[0092] This embodiment uses graded and refined decoding control based on confidence intervals to enable the model to automatically switch generation strategies under different reliability states. This avoids hallucination output under low confidence and prevents error solidification under high confidence, thereby effectively improving the factuality and credibility of the generated content without significantly increasing inference overhead.

[0093] Specifically, the method of adjusting the decoding parameters related to the text generation stream based on the preset divergent intervention strategy can be: increasing the temperature parameter by a preset fourth value, and inserting a preset self-questioning prompt word into the content generation stream to participate in content generation.

[0094] Specifically, increasing the temperature parameter by a preset fourth value flattens the softmax distribution, reduces the dominance of high-probability terms, and increases the sampling opportunities for low-probability but semantically reasonable alternatives, thus breaking the model's over-fixation of the current inference path. Simultaneously, inserting preset self-doubting prompts into the content generation stream not only introduces moderate uncertainty at the probabilistic level but also activates the model's internal verification mechanism at the semantic level, effectively intervening in overconfidence-driven illusions and improving the prudence and robustness of the generated content.

[0095] The preset fourth value can be pre-configured according to the task scenario and model behavior characteristics (e.g., 0.2-0.5) to ensure that the degree of divergence is controllable and to avoid introducing irrelevant noise or semantic drift.

[0096] Specifically, the preset self-questioning prompts are a set of predefined natural language instructions or guides used to stimulate the model's reflective ability, such as: "Is this statement accurate?", "Are there other possible explanations?", "Please verify the reliability of this information", etc.

[0097] The prompt can be dynamically appended to the end of the currently generated text (or injected into the context as a system message) and used as new conditional input in the next round of autoregressive generation. This guides the model to actively evaluate the rationality of its previous statements and, if necessary, self-correct or provide more cautious expressions. From the model's internal perspective, this is equivalent to introducing a virtual critical thinking agent, prompting the model to reassess the reliability, completeness, or logical consistency of previous statements when generating the next lexical term. In other words, by combining algorithmic injection of controllable uncertainty with semantic-level guiding signals, the model's self-questioning and corrective capabilities are stimulated, thereby suppressing the "high-confidence illusion" (i.e., the model outputs erroneous or fictitious content with high certainty). Specifically, the self-questioning prompts can be stored in the system prompt library and dynamically selected or combined according to the triggering scenario (such as detecting a high-confidence + ambiguous entity) to ensure that the guidance is targeted.

[0098] Specifically, before the step of inserting the preset self-questioning prompts into the content generation stream, the following can be further performed: obtain the current generated content of the large model, and generate the preset self-questioning prompts based on the current generated content through a preset prompt optimization model, wherein the preset prompt optimization model is obtained through supervised learning training based on preset context content, different adjustment modes, and different reflection prompts.

[0099] The preset prompt optimization model is a lightweight neural network, such as a small Transformer or MLP. Its input includes: the currently generated content (e.g., the current word or phrase fragment), the triggered adjustment mode identifier (first adjustment mode), and optional confidence state signals (e.g., policy entropy, transition word features, etc.). The output is a customized self-questioning prompt text, such as: "Is this conclusion reliable? Please re-evaluate in conjunction with authoritative sources" or "Are there other explanations? Please consider from the opposite perspective," etc.

[0100] Specifically, the preset prompt optimization model is trained offline through supervised learning. Its training data includes triples: input: historically generated output text and adjustment mode labels; target output: the most effective reflective prompt words in this context and mode, generated by manual annotation or a high-quality rule engine; training objective: to minimize the language modeling loss or semantic similarity loss between the generated prompt and the reference prompt.

[0101] This embodiment no longer relies on static, general self-questioning templates, but can dynamically synthesize reflection prompts with the highest targetedness and guiding effectiveness according to the current generation context and intervention intention, significantly improving the accuracy and effectiveness of the intervention.

[0102] Finally, the generated self-questioning prompt is inserted into the content generation stream, and participates in attention calculation and token prediction as the context for subsequent generation, thereby realizing semantic-level guidance of model behavior.

[0103] Reference Figure 4 and Figure 5 , this embodiment constructs a closed-loop control mechanism with real-time feedback and dynamic adjustment during the autoregressive generation process of a large model. Taking the generation of the t-th token as an example, after the t-th token is generated, the real-time confidence monitoring submodule immediately collects the current context information, calculates and outputs the current confidence signal. The current confidence signal is obtained by fusing multi-dimensional indicators such as policy entropy and transition word features, and is used to characterize the reliability state of the model before generating the next token.

[0104] The current confidence signal is compared with a preset target confidence interval [L, U], if Ct 1 < L, it is determined as low confidence risk and the focus mode is triggered; if Ct 1 > U, it is determined as overconfidence risk and the reflection mode is triggered; if L ≤ Ct 1 ≤ U, it is determined to be in an ideal operating state and no intervention is triggered. If the focus mode is triggered: a convergence operation is performed, for example, lowering the temperature parameter, reducing the top-p / top-k value, and compressing the sampling space; if the reflection mode is triggered: a divergence operation is performed, for example, increasing the temperature parameter, dynamically generating and inserting context-aware self-questioning prompts; if no adjustment mode is triggered: maintaining the default decoding parameters and no intervention is performed. All intervention operations are completed before generating the t-th token, ensuring that the regulation is forward-looking. The model performs forward inference based on the current context (including any inserted prompts) and the updated decoding parameters to generate the t-th token. The process iterates cyclically until the generation process ends (e.g., the maximum length is reached or a termination token is generated).

[0105] In this embodiment, through the above bidirectional dynamic adjustment mechanism, the model is continuously guided to operate near a sweet spot where confidence and prudence are balanced, avoiding generating hallucinated content due to excessive uncertainty and preventing solidification of wrong cognition due to false high confidence.

[0106] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description and will not be repeated hereafter. Based on this, the implementation method for determining the first adjustment mode according to the current confidence level and the preset confidence interval can be: Based on the current confidence level and the preset confidence interval, determine whether there is an anomaly in the current confidence state of the large model; if so, call the lightweight verification model and output an external verification signal through the lightweight verification model, wherein the lightweight verification model is obtained by supervised training based on a historical fact dataset of a specific domain; determine the first adjustment mode based on the external verification signal and the preset confidence interval.

[0107] This embodiment further introduces an external lightweight verification mechanism to enhance the ability to identify high-risk generated content.

[0108] Specifically, based on the current confidence level and the preset confidence interval, it is determined whether the current confidence state of the large model is abnormal. If the current confidence level is not within the preset confidence interval, it is determined that the current confidence state of the large model is abnormal and requires further processing; if the current confidence level is within the preset confidence interval, it is determined that the current confidence state of the large model is not abnormal and requires no processing.

[0109] When an anomaly is detected, a lightweight verification model is invoked for external verification. This verification model is configured as follows: Input: Partial text content (or key entity / claim fragments) that has already been generated; Output: External verification signal, which indicates the degree of consistency between the content and credible facts (the higher the value, the more credible).

[0110] The lightweight verification model can be a small discriminator trained on a domain-specific historical fact dataset (such as a medical guideline database, a legal text database, or a financial annual report corpus) through supervised learning. Examples include: a fact consistency classifier fine-tuned based on BERT; a truth-matching model for entity-attribute pairs; and a retrieval-enhanced similarity scoring module. This lightweight verification model is small in size and fast inference, completing verification within milliseconds, making it suitable for real-time generation scenarios.

[0111] Taking into account both the original confidence level Ct and the external verification signal V (V∈[0,1]), the first adjustment mode can be determined according to the following rules: if the current confidence level is less than the lower limit of the preset confidence interval and the external verification signal is less than the preset verification confidence threshold, it is determined to be a high-risk illusion, triggering the focus mode, and can enable forced retrieval or output blocking; wherein, the preset verification confidence threshold can be configured according to the domain sensitivity.

[0112] If the current confidence level is greater than the upper limit of the preset confidence interval, but the external verification signal is less than or equal to the current confidence level, it is judged as an overconfident illusion, triggering the reflection mode, inserting a questioning prompt and increasing the temperature to prompt correction; if the current confidence level is less than the lower limit of the preset confidence interval, but the external verification signal is greater than the preset verification confidence threshold, it is judged as a conservative estimate, and the default mode can be maintained or only slightly intervened to avoid excessive suppression of reasonable generation.

[0113] This embodiment improves the accuracy of hallucination risk assessment without excessively increasing latency by fusing the internal signal of the present invention with the output of a lightweight external verification model.

[0114] After the step of determining whether the current confidence state of the large model is abnormal based on the current confidence level and the preset confidence interval, the method further includes: If it exists, the hallucination-related data of the large model is recorded, wherein the hallucination-related data includes at least one of the current input context, current generated content, and current confidence state signal of the large model; the large model is fine-tuned according to the hallucination-related data, so as to generate content based on the fine-tuned large model.

[0115] If the current confidence state of the large model is determined to be abnormal, a set of hallucination-related data can be automatically recorded: the input context of the large model before generating the current word, i.e., the currently generated output text; the currently generated content (e.g., the current word or phrase fragment) generated by the large model in this context; the current confidence state signal corresponding to the large model (e.g., policy entropy value, transition word feature vector, fusion confidence Ct, etc.); external verification signals or user feedback markers, etc.

[0116] The above data can be used to construct a negative sample training set or preference alignment data, and the large model can be continuously optimized using methods such as supervised fine-tuning, reinforcement learning based on human feedback, and contrastive learning. Subsequent content generation can be based on the optimized large model, thereby reducing the probability of the same hallucination recurring at the source and achieving closed-loop self-evolution.

[0117] In actual deployment, fine-tuning can be achieved through online incremental learning or periodic batch update modes.

[0118] This embodiment uses an automatic capture of hallucination data and a model self-optimization mechanism to not only correct hallucinations during use, but also to enable the model to learn from the root cause in which aspects it should be more cautious, thereby achieving continuous model optimization.

[0119] Specifically, before the step of determining the first adjustment mode based on the current confidence level and the preset confidence interval, the method further includes: The generation performance of the large model is evaluated based on a preset set of validation questions, wherein the generation performance includes a trade-off between accuracy and rejection rate. Based on the generation performance, a preset confidence interval is output using a confidence optimization algorithm, wherein the confidence optimization algorithm is used to search for a target confidence interval that optimizes the trade-off.

[0120] It should be noted that the preset verification question set contains typical query samples with known standard answers, covering the key semantics and fact types of the target task.

[0121] For each question, a large model is run to generate content. The generation performance of the large model is calculated, which is a trade-off between accuracy and rejection rate. Accuracy is the proportion of generated content that matches the standard answer; rejection rate is the proportion of the model actively refusing to answer or being blocked by intervention mechanisms in a low-confidence state.

[0122] Based on the above-mentioned performance, a confidence optimization algorithm is invoked, using a pre-set confidence interval [L, U] as the optimization variable, to search for a target confidence interval that maximizes the trade-off metric. Specifically, searching for the target confidence interval that maximizes the trade-off metric through the confidence optimization algorithm can involve selecting the interval with the highest performance score via grid search, or constructing a surrogate model based on historical evaluation results using Bayesian optimization to efficiently explore high-yield regions.

[0123] The optimized target confidence interval is used as the preset confidence interval for this task, and is used for pattern decision-making in the subsequent real-time generation process.

[0124] Through this self-calibration mechanism, this embodiment upgrades the preset confidence interval from a static configuration parameter to a data-driven dynamic optimal solution, which significantly improves the robustness, portability and task adaptability of the entire hallucination suppression.

[0125] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the hallucination suppression method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0126] This application also provides a model hallucination suppression device, please refer to... Figure 6 The model hallucination suppression device includes: The acquisition module 10 is used to acquire the current confidence state signal of the large model before the target content unit is generated from the content generation stream of the large model; Adjustment module 20 is used to adjust control parameters related to the content generation stream based on the current confidence state signal and the preset ideal confidence state; The guidance module 30 is used to guide the generation behavior of the target content unit based on the adjusted control parameters, so that the confidence state signal of the large model is maintained at the preset ideal confidence state, thereby suppressing the illusion of the large model.

[0127] In one embodiment, the content generation stream includes a text generation stream, the control parameters include decoding parameters, and the adjustment module 20 includes: The first determining submodule is used to determine the current confidence level of the large model based on the current confidence state signal; The second determining submodule is used to determine the first adjustment mode based on the current confidence level and the preset confidence interval, wherein the preset confidence interval is used to define the preset ideal confidence state; The first adjustment submodule is used to adjust the decoding parameters related to the text generation stream according to the first adjustment mode.

[0128] In one embodiment, the first adjustment mode includes a focus mode and a reflection mode, and the second determining submodule includes: The first determining unit is configured to determine the first adjustment mode as the focusing mode when the current confidence level is lower than the lower limit of the preset confidence level interval; The second determining unit is used to determine the first adjustment mode as the reflection mode when the current confidence level is higher than the upper limit of the preset confidence level interval.

[0129] In one embodiment, the first adjustment submodule includes: The first adjustment unit is used to adjust the decoding parameters related to the text generation stream based on a preset convergence intervention strategy when the first adjustment mode is the focus mode. The preset convergence intervention strategy is used to suppress low-probability words. The second adjustment unit is used to adjust the decoding parameters related to the text generation stream based on a preset divergent intervention strategy when the first adjustment mode is the reflection mode. The preset divergent intervention strategy is used to inject uncertainty to guide the model to self-correct.

[0130] In one embodiment, the decoding parameters include at least one of a temperature parameter, a k value of Top-k, and a p value of Top-p; The first adjustment unit includes at least one of the following: The first adjustment subunit is used to lower the temperature parameter by a preset first value, wherein the temperature parameter is used to control the smoothness of the output probability distribution. The second adjustment subunit is used to lower the k value in Top-k by a preset second value; The third adjustment subunit is used to lower the p value in Top-p by a preset third value.

[0131] In one embodiment, the second adjustment unit includes: The adjustment subunit is used to increase the temperature parameter by a preset fourth value and insert a preset self-questioning prompt into the content generation stream to participate in content generation.

[0132] In one embodiment, prior to the step of inserting a preset self-questioning prompt into the content generation stream, the apparatus further includes: The generation module is used to obtain the current generated content of the large model, and generate the preset self-questioning prompt words based on the current generated content through the preset prompt optimization model. The preset prompt optimization model is obtained by supervised learning based on preset context content, different adjustment modes and different reflection prompts.

[0133] In one embodiment, the current confidence state signal includes at least one of output layer features and intermediate layer feature variance, wherein the output layer features include policy entropy and transition word features; the first determining submodule includes at least one of the following: The normalization unit is used to normalize the policy entropy or intermediate layer feature variance to obtain a normalized value, and the normalized value is used as the current confidence of the large model. The standardization unit is used to standardize the transition word features to obtain normalized transition word features. Based on preset weight coefficients, the strategy entropy and the normalized transition word features are weighted and summed to obtain the current confidence of the large model.

[0134] In one embodiment, the second determining submodule includes: The judgment unit is used to determine whether there is an anomaly in the current confidence state of the large model based on the current confidence level and the preset confidence interval; The output unit is used to call the lightweight verification model if it exists, and output an external verification signal through the lightweight verification model, wherein the lightweight verification model is obtained by supervised training based on a historical fact dataset of a specific domain. The fourth determining unit is used to determine the first adjustment mode based on the external verification signal and the preset confidence interval.

[0135] In one embodiment, after the step of determining whether the current confidence state of the large model is abnormal based on the current confidence level and a preset confidence interval, the device further includes: A recording module is used to record, if present, hallucination-related data of the large model, wherein the hallucination-related data includes at least one of the current input context, current generated content, and current confidence state signal of the large model; The fine-tuning module is used to fine-tune the large model based on the hallucination-related data, so as to generate content based on the fine-tuned large model.

[0136] In one embodiment, before the step of determining the first adjustment mode based on the current confidence level and a preset confidence interval, the device further includes: The evaluation module is used to evaluate the generation performance of the large model based on a preset set of verification questions, wherein the generation performance includes a trade-off between accuracy and rejection rate. The output module is used to output the preset confidence interval based on the generated performance using a confidence optimization algorithm, wherein the confidence optimization algorithm is used to search for a target confidence interval that optimizes the trade-off index.

[0137] In one embodiment, when the content generation stream is a text generation stream, the target content unit includes target words, and the acquisition module 10 includes: The acquisition submodule is used to acquire the output probability distribution of the previous word corresponding to the target word, as well as the output text currently generated by the large model; The calculation submodule is used to calculate the policy entropy of the output probability distribution; The third determination submodule is used to scan the output text according to a preset transition word dictionary and determine the transition word features in the output text. The transition word features are used to reflect the inference state of the model.

[0138] In one embodiment, the adjustment module 20 includes: The output submodule is used to output a second adjustment mode through a reinforcement learning agent based on the current confidence state signal. The reinforcement learning agent is trained by taking the historical confidence state signal as the state input, the adjustment mode as the action output, and the validity evaluation signal of the generated content as the basis. The validity evaluation signal of the generated content is used to calibrate the boundary of the preset ideal confidence state. The second adjustment submodule is used to adjust the control parameters related to the content generation stream based on the second adjustment mode.

[0139] The model hallucination suppression device provided in this application, employing the model hallucination suppression method in the above embodiments, can solve the technical problem that post-verification and correction methods are insufficient to fundamentally suppress the generation of model hallucinations. Compared with the prior art, the beneficial effects of the model hallucination suppression device provided in this application are the same as those of the model hallucination suppression method provided in the above embodiments, and other technical features in the model hallucination suppression device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0140] This application provides a model hallucination suppression device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the model hallucination suppression method in Embodiment 1 above.

[0141] The following is for reference. Figure 7 The diagram illustrates a structural schematic suitable for implementing the model hallucination suppression device in the embodiments of this application. The model hallucination suppression device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, tablets, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital televisions and desktop computers. Figure 7 The illustrated hallucination suppression device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0142] like Figure 7 As shown, the model hallucination suppression device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the model hallucination suppression device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the model hallucination suppression device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows model hallucination suppression devices with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0143] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0144] The model hallucination suppression device provided in this application, employing the model hallucination suppression method in the above embodiments, can solve the technical problem that post-verification and correction methods are insufficient to fundamentally suppress the generation of model hallucinations. Compared with the prior art, the beneficial effects of the model hallucination suppression device provided in this application are the same as those of the model hallucination suppression method provided in the above embodiments, and other technical features of this model hallucination suppression device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0145] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0146] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0147] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the model illusion suppression method in the above embodiments.

[0148] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0149] The aforementioned computer-readable storage medium may be included in the model hallucination suppression device; or it may exist independently and not assembled into the model hallucination suppression device.

[0150] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the model illusion suppression device, enable the model illusion suppression device to: have independent control.

[0151] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0153] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0154] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described model hallucination suppression method. This solves the technical problem that post-verification and correction methods are insufficient to fundamentally suppress the generation of model hallucinations. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the model hallucination suppression method provided in the above embodiments, and will not be elaborated upon here.

[0155] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the model illusion suppression method described above.

[0156] The computer program product provided in this application can solve the technical problem that post-verification and correction methods are insufficient to fundamentally suppress the generation of model illusions. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the model illusion suppression method provided in the above embodiments, and will not be repeated here.

[0157] All acquisition of signals, information, or actions in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization of the relevant device owner.

[0158] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A method for suppressing model hallucinations, characterized in that, The method includes: Before the target content unit is generated from the content generation stream of the large model, the current confidence state signal of the large model is obtained; Based on the current confidence state signal and the preset ideal confidence state, adjust the control parameters related to the content generation stream; Based on the adjusted control parameters, the generation behavior of the target content unit is guided, so that the confidence state signal of the large model is maintained at the preset ideal confidence state, thereby suppressing the illusion of the large model.

2. The method as described in claim 1, characterized in that, The content generation stream includes a text generation stream, the control parameters include decoding parameters, and the step of adjusting the control parameters related to the content generation stream based on the current confidence state signal and a preset ideal confidence state includes: Based on the current confidence state signal, determine the current confidence level of the large model; Based on the current confidence level and the preset confidence interval, a first adjustment mode is determined, wherein the preset confidence interval is used to define the preset ideal confidence state; According to the first adjustment mode, the decoding parameters related to the text generation stream are adjusted.

3. The method as described in claim 2, characterized in that, The first adjustment mode includes a focusing mode and a reflective mode. The step of determining the first adjustment mode based on the current confidence level and a preset confidence interval includes: When the current confidence level is lower than the lower limit of the preset confidence level interval, the first adjustment mode is determined to be the focus mode; When the current confidence level is higher than the upper limit of the preset confidence interval, the first adjustment mode is determined to be the reflection mode.

4. The method as described in claim 3, characterized in that, The step of adjusting the decoding parameters related to the text generation stream according to the first adjustment mode includes: When the first adjustment mode is the focus mode, the decoding parameters related to the text generation stream are adjusted based on a preset convergence intervention strategy. The preset convergence intervention strategy is used to suppress low-probability words. When the first adjustment mode is the reflection mode, the decoding parameters related to the text generation stream are adjusted based on a preset divergent intervention strategy. The preset divergent intervention strategy is used to inject uncertainty to guide the model to self-correct.

5. The method as described in claim 4, characterized in that, The decoding parameters include at least one of the following: temperature parameter, k value of Top-k, and p value of Top-p; The step of adjusting the decoding parameters related to the text generation stream based on a preset convergence intervention strategy includes at least one of the following: The temperature parameter is lowered by a preset first value, wherein the temperature parameter is used to control the smoothness of the output probability distribution; Lower the k value in Top-k by a preset second value; The p value in Top-p is lowered by a preset third value.

6. The method as described in claim 5, characterized in that, The step of adjusting the decoding parameters related to the text generation stream based on a preset divergent intervention strategy includes: The temperature parameter is increased to a preset fourth value, and a preset self-questioning prompt is inserted into the content generation stream to participate in content generation.

7. The method as described in claim 6, characterized in that, Before the step of inserting preset self-questioning prompts into the content generation stream, the method further includes: The current generated content of the large model is obtained, and the preset self-questioning prompt words are generated based on the current generated content by the preset prompt optimization model. The preset prompt optimization model is obtained by supervised learning based on preset context content, different adjustment modes and different reflection prompts.

8. The method as described in claim 2, characterized in that, The current confidence state signal includes at least one of the output layer features and the variance of the intermediate layer features, wherein the output layer features include policy entropy and transition word features; the step of determining the current confidence of the large model based on the current confidence state signal includes at least one of the following: The policy entropy or intermediate layer feature variance is normalized to obtain a normalized value, which is then used as the current confidence level of the large model. The transition word features are standardized to obtain normalized transition word features. Based on preset weight coefficients, the policy entropy and the normalized transition word features are weighted and summed to obtain the current confidence of the large model.

9. The method as described in claim 2, characterized in that, The step of determining the first adjustment mode based on the current confidence level and the preset confidence interval includes: Based on the current confidence level and the preset confidence interval, determine whether there is an anomaly in the current confidence state of the large model; If it exists, the lightweight verification model is invoked, and the external verification signal is output through the lightweight verification model, wherein the lightweight verification model is obtained by supervised training based on a historical fact dataset of a specific domain; The first adjustment mode is determined based on the external verification signal and the preset confidence interval.

10. The method as described in claim 9, characterized in that, After the step of determining whether the current confidence state of the large model is abnormal based on the current confidence level and the preset confidence interval, the method further includes: If it exists, then record the hallucination-related data of the large model, wherein the hallucination-related data includes at least one of the current input context, current generated content and current confidence state signal of the large model; Based on the hallucination-related data, the large model is fine-tuned to generate content based on the fine-tuned large model.

11. The method as described in claim 2, characterized in that, Before the step of determining the first adjustment mode based on the current confidence level and the preset confidence interval, the method further includes: The generation performance of the large model is evaluated based on a preset set of verification questions, wherein the generation performance includes a trade-off between accuracy and abandonment rate. Based on the generated performance, the preset confidence interval is output through a confidence optimization algorithm, wherein the confidence optimization algorithm is used to search for a target confidence interval that optimizes the trade-off index.

12. The method as described in claim 1, characterized in that, When the content generation stream is a text generation stream, the target content unit includes target lexical units, the current confidence state signal includes policy entropy and transition word features, and the step of obtaining the current confidence state signal of the large model includes: Obtain the output probability distribution of the previous word corresponding to the target word, and the output text currently generated by the large model; Calculate the policy entropy of the output probability distribution; The output text is scanned according to a preset transition word dictionary to determine the transition word features in the output text. The transition word features are used to reflect the inference state of the model.

13. The method as described in claim 1, characterized in that, The step of adjusting the control parameters related to the content generation stream based on the current confidence state signal and the preset ideal confidence state includes: Based on the current confidence state signal, a second adjustment mode is output by the reinforcement learning agent. The reinforcement learning agent is trained by taking the historical confidence state signal as the state input, the adjustment mode as the action output, and the validity evaluation signal of the generated content as the basis. The validity evaluation signal of the generated content is used to calibrate the boundary of the preset ideal confidence state. Based on the second adjustment mode, the control parameters related to the content generation stream are adjusted.

14. A model hallucination suppression device, characterized in that, The device includes: The acquisition module is used to acquire the current confidence state signal of the large model before the target content unit is generated from the content generation stream of the large model; The adjustment module is used to adjust the control parameters related to the content generation stream based on the current confidence state signal and the preset ideal confidence state. The guidance module is used to guide the generation behavior of the target content unit based on the adjusted control parameters, so that the confidence state signal of the large model is maintained at the preset ideal confidence state, thereby suppressing the illusion of the large model.

15. A model hallucination suppression device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the model hallucination suppression method as described in any one of claims 1 to 13.

16. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the model hallucination suppression method as described in any one of claims 1 to 13.

17. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the model hallucination suppression method as described in any one of claims 1 to 13.

Images