Noise adaptive method and apparatus for intelligent question answering model

Abstract

This disclosure provides a noise adaptation method and apparatus for intelligent question answering models, relating to the field of intelligent question answering technology. The method includes: acquiring an intelligent question answering model and multiple question samples; performing multi-round intelligent question answering training on the intelligent question answering model based on the multiple question samples; gradually increasing the proportion or intensity of injected noise during the multi-round intelligent question answering training process; analyzing whether the intelligent question answering model identifies and corrects pre-injected noise in each round of question answering training using a teacher model, and determining a noise robustness reward based on whether the intelligent question answering model accurately identifies noise and takes correct measures; wherein, the noise robustness reward is used in conjunction with task completion rewards to jointly execute a reinforcement learning gradient optimization strategy to update the parameters of the intelligent question answering model. Embodiments of this disclosure can significantly improve the adaptability, robustness, and generalization ability of intelligent question answering models in real noisy environments.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent question answering technology, and more particularly to a noise adaptation method and electronic device for intelligent question answering models. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this disclosure as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] In the field of intelligent question answering technology, ensuring that models (such as large models) can provide accurate and reliable answers to users’ real, diverse queries that often contain noise (such as ambiguous expressions, contradictory information, or errors returned by external knowledge sources) is one of the core challenges for achieving widespread application in this field.

[0004] Current technologies primarily rely on manually defined rules or independent post-processing modules to address this type of noise. This not only makes it difficult for the system to adapt to rapidly changing user questions and emerging noise types, but also sacrifices the real-time interactive experience sought by intelligent question-answering systems due to the additional processing steps. More fundamentally, current solutions fail to internalize noise processing capabilities within the model itself, resulting in insufficient robustness in complex and dynamic real-world question-answering scenarios. This leads to a significant decrease in the quality and stability of responses, hindering the reliable deployment and performance of intelligent question-answering systems in open domains and scenarios with high real-time requirements (such as online customer service and real-time information assistants). Summary of the Invention

[0005] The purpose of this disclosure is to provide a noise adaptation method and electronic device for intelligent question answering models, which can significantly improve the adaptability, robustness and generalization ability of intelligent question answering models in real noisy environments.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] This disclosure provides a noise adaptation method for an intelligent question-answering model, comprising: acquiring an intelligent question-answering model and multiple question samples; performing multi-round intelligent question-answering training on the intelligent question-answering model based on the multiple question samples; during the multi-round intelligent question-answering training, increasing the proportion or intensity of injected noise in stages according to a preset stepping rule, injecting at least one preset type of noise into the data processed by the intelligent question-answering model; analyzing whether the intelligent question-answering model identifies and corrects the pre-injected noise in each round of question-answering training through a teacher model, and determining a noise robustness reward based on whether the intelligent question-answering model accurately identifies the noise and takes correct measures; wherein, the noise robustness reward is used to jointly execute a reinforcement learning gradient optimization strategy in conjunction with task completion rewards to update the parameters of the intelligent question-answering model.

[0008] In some embodiments, according to a preset stepping pattern, the proportion or intensity of injected noise is increased in stages to inject at least one preset type of noise into the data processed by the intelligent question-answering model, including: obtaining the current training round of the intelligent question-answering model; calculating the noise injection probability corresponding to the current training round according to the ratio of the current training round to the preset total training rounds, based on a preset functional relationship; wherein, the preset functional relationship includes a linear function, an sigmoid function, or a step function that monotonically increases with the ratio; and injecting at least one preset type of noise into the data processed by the intelligent question-answering model based on the noise injection probability corresponding to the current training round.

[0009] In some embodiments, the noise injection probability corresponding to the current training round is calculated according to a preset functional relationship based on the ratio of the current training round to the preset total training rounds, including: if the ratio is less than a first threshold, controlling the noise injection probability to be in a first probability interval, and increasing it linearly or stepwise with the training progress; if the ratio is greater than or equal to the first threshold and less than a second threshold, controlling the noise injection probability to be in a second probability interval, and increasing it linearly or stepwise with the training progress; wherein the starting value of the second probability interval is not lower than the ending value of the first probability interval; if the ratio is greater than or equal to the second threshold, maintaining the noise injection probability at a target probability value used to simulate a real noise environment, wherein the ending value of the second probability interval does not exceed the target probability value; wherein the first threshold is greater than the second threshold.

[0010] In some embodiments, according to a preset stepping pattern, the proportion or intensity of injected noise is increased in stages to inject at least one preset type of noise into the data processed by the intelligent question answering model, including: determining whether to perform noise injection in the current training round based on the noise injection probability; and / or, based on the noise injection probability, determining at least one noise type as a target noise type among the at least one preset type, and injecting at least one preset type of noise into the data processed by the intelligent question answering model.

[0011] In some embodiments, the at least one preset type of noise includes at least one of tool noise, model noise, input noise, and training data noise; the tool noise originates from errors generated when the intelligent question answering model calls external tools; the model noise originates from errors generated by the intelligent question answering model's own reasoning; the input noise originates from errors carried in the question samples; and the training data noise originates from errors in the labeled data used to train the intelligent question answering model.

[0012] In some embodiments, the plurality of question samples includes a first question sample or a second question sample; wherein, injecting at least one preset type of noise into the data processed by the intelligent question answering model includes at least one of the following methods: when the intelligent question answering model requests data from an external tool, returning the tool noise to the intelligent question answering model, the tool noise including at least one of format error noise, numerical error noise, or tool call failure noise; obtaining the pre-reasoning generated by the intelligent question answering model for the first question sample; injecting errors into the pre-reasoning to obtain erroneous pre-reasoning; sending the erroneous pre-reasoning back to the intelligent question answering model so that the intelligent question answering model continues to perform subsequent reasoning for the first question sample based on the erroneous pre-reasoning and finally outputs the answer to the first question sample; modifying or replacing the annotation information associated with the second question sample; inserting irrelevant, contradictory, or ambiguous interference information into the original content of the second question sample.

[0013] In some embodiments, the multi-round intelligent question-answering training includes target intelligent question-answering training; wherein, through a teacher model, the system analyzes whether the intelligent question-answering model identifies and corrects pre-injected noise in each round of question-answering training, and determines a noise robustness reward based on whether the intelligent question-answering model accurately identifies noise and takes correct measures, including: analyzing the question-answering behavior of the intelligent question-answering model during the target intelligent question-answering training process through the teacher model; if noise is injected during the target intelligent question-answering training process and is identified by the intelligent question-answering model, then the noise robustness reward is set to a first value; if... If no noise is injected during the training of the target intelligent question answering system and the intelligent question answering model does not identify the noise, then the noise robustness reward is set to a second value. If noise is injected during the training of the target intelligent question answering system but is not identified by the intelligent question answering model, then the noise robustness reward is set to a third value, which is less than the first value and less than the second value. If no noise is injected during the training of the target intelligent question answering system but is identified as having noise by the intelligent question answering model, then the noise robustness reward is set to a fourth value, which is less than the first value and less than the second value.

[0014] This disclosure provides a noise adaptation device for an intelligent question-answering model, including: a sample acquisition module, a noise injection module, and a noise response capability analysis module.

[0015] The sample acquisition module is used to acquire the intelligent question-answering model and multiple question samples. The noise injection module can be used to perform multiple rounds of intelligent question-answering training on the intelligent question-answering model based on the multiple question samples. During the multiple rounds of intelligent question-answering training, the proportion or intensity of injected noise is increased in stages according to a preset stepping rule, injecting at least one preset type of noise into the data processed by the intelligent question-answering model. The noise response capability analysis module can be used to analyze whether the intelligent question-answering model identifies and corrects the pre-injected noise in each round of question-answering training using a teacher model, and determine the noise robustness reward based on whether the intelligent question-answering model accurately identifies the noise and takes the correct measures. The noise robustness reward is used to combine the task completion reward with the joint execution of a reinforcement learning gradient optimization strategy to update the parameters of the intelligent question-answering model.

[0016] This disclosure provides an electronic device comprising: a memory and a processor; the memory for storing computer program instructions; and the processor for calling the computer program instructions stored in the memory to implement the noise adaptation method for an intelligent question-answering model as described above.

[0017] This disclosure provides a computer-readable storage medium storing computer program instructions to implement the noise adaptation method for intelligent question-answering models as described in any of the preceding embodiments.

[0018] This disclosure provides a computer program product or computer program that includes computer program instructions stored in a computer-readable storage medium. The computer program instructions are read from the computer-readable storage medium, and a processor executes the computer program instructions to implement the aforementioned noise adaptation method for an intelligent question-answering model.

[0019] The noise adaptation method, apparatus, electronic device, computer-readable storage medium, and computer program product for intelligent question answering models provided in this disclosure progressively inject preset noise during multiple rounds of training and combine it with a noise robustness reward mechanism based on a teacher model. This enables the intelligent question answering model to actively learn to identify and process various types of noise during the training process, thereby significantly improving the robustness, adaptability, and generalization ability of the model in real noisy environments and realizing the transformation from relying on external post-processing to endogenous noise immunity.

[0020] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0022] Figure 1 A schematic diagram of a scenario is shown that can be applied to a noise adaptation method or a noise adaptation device for an intelligent question answering model, which can be used in embodiments of this disclosure.

[0023] Figure 2 This is a flowchart illustrating a noise adaptation method for an intelligent question-answering model according to an exemplary embodiment.

[0024] Figure 3 This is a flowchart illustrating a noise injection method according to an exemplary embodiment.

[0025] Figure 4 This is a flowchart illustrating a noise injection method according to an exemplary embodiment.

[0026] Figure 5This is a flowchart illustrating a noise injection method according to an exemplary embodiment.

[0027] Figure 6 This is a flowchart illustrating a noise injection method according to an exemplary embodiment.

[0028] Figure 7 This is a flowchart illustrating a noise robustness reward determination method according to an exemplary embodiment.

[0029] Figure 8 This is a flowchart illustrating a noise adaptation method for an intelligent question-answering model according to an exemplary embodiment.

[0030] Figure 9 This is a block diagram illustrating a noise adaptation device for an intelligent question-answering model according to an exemplary embodiment.

[0031] Figure 10 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0033] Those skilled in the art will recognize that embodiments of this disclosure can be a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0034] The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0035] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0036] The accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus omitting repeated descriptions of them. Some block diagrams shown in the drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0037] The flowchart shown in the accompanying drawings is merely illustrative and does not necessarily include all content and steps, nor does it require execution in the described order. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0038] In the description of this disclosure, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences; the terms "contains," "includes," and "has" are used to indicate an open-ended meaning of inclusion and refer to the existence of additional elements / components / etc. besides those listed.

[0039] This disclosure embodiment can be implemented by a terminal and / or a server. The terminal can obtain data from a computer device and display that data. The computer device can interact with the terminal, and can be a server hosting the application, or it can belong to the terminal (i.e., the terminal's backend), etc., without limitation.

[0040] The terminal can be a mobile phone, a laptop computer, or a playback device in a vehicle, etc., without limitation. The terminal can be considered a playback device in a vehicle, and it can display the target application. The terminal is only one example of the devices listed; the terminal in this disclosure is not limited to the listed devices. The target application in this disclosure can be any application capable of displaying multimedia information.

[0041] It is understood that the terminal mentioned in the embodiments of this disclosure can be a computer device, including but not limited to a terminal or a server. In other words, the computer device can be a server or a terminal, or a system composed of a server and a terminal. The terminal mentioned above can be an electronic device, including but not limited to mobile phones, tablets, desktop computers, laptops, handheld computers, in-vehicle devices, augmented reality / virtual reality (AR / VR) devices, head-mounted displays, smart TVs, wearable devices, smart speakers, digital cameras, webcams, and other mobile internet devices (MIDs) with network access capabilities, or terminals in scenarios such as trains, ships, and flights.

[0042] The servers mentioned above can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, vehicle-road cooperation, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0043] Optionally, the data involved in the embodiments of this disclosure may be stored in a computer device or may be stored based on cloud storage technology, without limitation.

[0044] With the deep application of large models in the field of intelligent question answering, the models must directly process query requests from real users that contain diverse noise. This noise includes not only the ambiguous, contradictory, or irrelevant information in the user input itself, but also the errors or incomplete data that external tools (such as search engines and database query tools) that the question answering process relies on may return, as well as the logical biases that the model itself may accumulate in multiple rounds of dialogue.

[0045] In existing technologies, mainstream solutions to this type of noise problem mostly rely on manual intervention or post-processing mechanisms. Specifically, the following methods are typically used: 1. Manual Rule Validation: A rule base or validation script is pre-written to address common error types (such as specific data format errors, inconsistent indicator definitions, etc.). After the model outputs results, these rules are run to correct or filter the results. However, this method heavily relies on manually summarized rules, requiring continuous maintenance and updates to the rule base for the ever-evolving error types, resulting in high costs. Furthermore, its generalization ability is limited, making it difficult to handle novel error scenarios that are unseen or involve complex combinations.

[0046] 2. Post-processing error correction mechanism: After the model completes the entire inference process, an independent "post-processing" module is added. This module may be based on another model or rule system to re-analyze and correct the initial output. A significant drawback of this approach is its poor real-time performance. The additional processing steps inevitably introduce additional computational and communication delays, making it difficult for the system to meet the demands of real-time analysis with high response speed requirements (such as online customer service, real-time business insights, etc.).

[0047] 3. Lack of intrinsic adaptability: Both rule validation and post-processing are external "patch" strategies. The model itself does not learn how to cope with these noises during training, and its core reasoning ability is not specifically enhanced. Therefore, when environmental instability intensifies or new noise patterns emerge, the model's performance is prone to significant decline, lacking the ability to adapt to dynamically changing environments.

[0048] The aforementioned methods are ill-suited to the openness, dynamism, and real-time nature of questions in intelligent question-answering scenarios. Manual rules cannot exhaustively cover all possible "imperfect questions" from users or various abnormal results returned by the tool, while post-processing mechanisms disrupt the smoothness and immediacy of intelligent question-answering interactions, severely impacting user experience. More importantly, existing solutions fail to enable intelligent question-answering models to learn and internalize the ability to identify and respond to noise during the training phase. This makes it difficult to guarantee the accuracy, stability, and robustness of responses in complex and ever-changing real-world interactive environments, hindering the reliable deployment of intelligent question-answering systems in demanding commercial scenarios (such as real-time customer service and dynamic decision support). Therefore, there is an urgent need for an endogenous enhancement method for the intelligent question-answering field that enables models to proactively adapt to noise during training, fundamentally improving their performance and reliability in practical applications.

[0049] To address the aforementioned issues, this application employs a course learning strategy and noise-robust rewards to enable the intelligent question-answering model to gradually adapt to various types of noise during the training phase. A reward mechanism incentivizes the model to actively identify and correct multiple errors. This method requires no manual rules, exhibits higher adaptability and real-time performance, and can generalize to various tools and scenarios.

[0050] The exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0051] Figure 1 A schematic diagram of a scenario is shown that can be applied to a noise adaptation method or a noise adaptation device for an intelligent question answering model, which can be used in embodiments of this disclosure.

[0052] Please refer to Figure 1 The diagram illustrates an implementation environment provided by an exemplary embodiment of this disclosure.

[0053] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0054] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, desktop computers, wearable devices, virtual reality devices, smart home devices, etc.

[0055] Server 105 can be a server that provides various services, such as a backend management server that supports the devices operated by users using terminal devices 101, 102, and 103. The backend management server can analyze and process received requests and other data, and feed the processing results back to the terminal devices.

[0056] A server can be a standalone physical server, a server cluster or a distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This disclosure does not impose any restrictions on this.

[0057] Server 105 may, for example, acquire an intelligent question-answering model and multiple question samples; server 105 may, for example, perform multiple rounds of intelligent question-answering training on the intelligent question-answering model based on the multiple question samples; during the multiple rounds of intelligent question-answering training, at least one preset type of noise is injected into the input and / or interaction stages of the intelligent question-answering model in a progressively increasing manner; server 105 may, for example, analyze the question-answering behavior of the intelligent question-answering model in each round of intelligent question-answering training through a teacher model, and determine the noise robustness reward corresponding to each round of intelligent question-answering training based on the analysis results; wherein, the noise robustness reward is used to quantify the intelligent question-answering model's ability to identify and process noise; wherein, the noise robustness reward is used in conjunction with the task completion reward to jointly execute a reinforcement learning gradient optimization strategy to drive the parameter update of the intelligent question-answering model.

[0058] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Server 105 can be a single physical server or a combination of multiple servers. Depending on actual needs, it can have any number of terminal devices, networks, and servers.

[0059] Figure 2 This is a flowchart illustrating a noise adaptation method for an intelligent question-answering model according to an exemplary embodiment. The method provided in this disclosure can be executed by any electronic device with computing power; for example, the method can be implemented by the aforementioned... Figure 1 The execution can be performed by a server or terminal device in the embodiments, or it can be performed by both a server and a terminal device. In the following embodiments, the server is used as the execution subject for illustration, but this disclosure is not limited to this.

[0060] Reference Figure 2 The noise adaptation method for intelligent question answering models provided in this disclosure may include the following steps.

[0061] Step S202: Obtain the intelligent question answering model and multiple question samples.

[0062] The aforementioned intelligent question answering model can refer to various large language models (LLMs) applied in the field of intelligent question answering, including but not limited to large text models, large speech models, large image models, and multimodal large models, etc. This application does not make specific limitations on this.

[0063] Question samples may include question texts used for model training and their corresponding reference answers or labeled data.

[0064] Step S204: Based on multiple question samples, perform multiple rounds of intelligent question answering training on the intelligent question answering model; during the multiple rounds of intelligent question answering training, according to the preset stepping rules, increase the proportion or intensity of injected noise in stages, and inject at least one preset type of noise into the data processed by the intelligent question answering model.

[0065] In some embodiments, the noise of at least one preset type may include at least one of tool noise, model noise, input noise, and training data noise. Tool noise originates from errors generated when the intelligent question-answering model calls external tools; model noise originates from errors generated by the intelligent question-answering model's own reasoning; input noise originates from errors carried in the question samples; and training data noise originates from errors in the labeled data used to train the intelligent question-answering model.

[0066] Below, this application will provide a detailed explanation of various noises.

[0067] In some embodiments, tool noise can include at least three categories: format noise—such as errors in the format of Markdown tables returned by the tool, missing column names, changes in null value symbols, and messy web page formats returned by browser tools; numerical noise—such as data returned by data extraction tools not matching the request, time ranges inconsistent with the query request, omission of important indicators, confusion in indicator definitions, and errors in content returned by search tools containing irrelevant web pages; and tool call failure—errors such as tool call failure due to communication delays.

[0068] In some embodiments, model noise can be noise generated by the model itself, which makes some incorrect assertions in the preceding reasoning steps, and subsequent reasoning steps may be affected by the incorrect assertions in the preceding steps.

[0069] In some embodiments, input noise can be noise in the user input, such as input containing irrelevant information, contradictory instructions, unclear meaning, etc.

[0070] In some embodiments, training data noise can be inherent errors in the training data. For example, when training using DPO (Direct Preference Optimization) or SFT (Supervised Fine-Tuning) algorithms, the training data may contain errors, such as incorrect preference labeling or illusion errors in the response data.

[0071] This embodiment simulates common error patterns in the real training and model application stages by precisely defining various noise types.

[0072] In some embodiments, the intelligent question-answering model can be continuously optimized using question samples through multiple rounds of iterative training. During the training process, the intensity of noise injection can be gradually increased—initially a small amount or simple noise is injected, and as the training rounds increase, the probability, complexity, and diversity of noise are gradually increased. Noise is also selectively introduced into the model input stage (such as question text contamination) and / or the interaction stage between the model and external tools and the environment (such as simulation tools returning erroneous data), so that the model can gradually adapt to various interferences in the real environment.

[0073] In some embodiments, the noise of at least one preset type described above may include tool noise. Below, this application will explain and illustrate how to inject tool noise into the input and / or interaction stages of an intelligent question-answering model.

[0074] In some embodiments, when the intelligent question answering model requests data from an external tool, tool noise may be returned to the intelligent question answering model. Tool noise may include at least one of format error noise, numerical error noise, or tool call failure noise.

[0075] In some embodiments, the aforementioned plurality of problem samples may include a first problem sample, and the aforementioned at least one preset type of noise includes model noise. Below, this application will use the first problem sample as an example to explain how to inject model noise.

[0076] In some embodiments, injecting at least one preset type of noise into the input and / or interaction stages of the intelligent question answering model in processing the first question sample may include the following steps: obtaining the preceding reasoning generated by the intelligent question answering model for the first question sample; injecting errors into the preceding reasoning to obtain preceding reasoning containing errors; and feeding back the preceding reasoning containing errors to the intelligent question answering model so that the intelligent question answering model can continue to perform subsequent reasoning for the first question sample based on the preceding reasoning containing errors, and finally output the answer to the first question sample.

[0077] The above process simulates the situation where an intelligent question-answering model makes mistakes during its own reasoning: First, the preceding reasoning content generated by the model for the current question is obtained, and then logical or factual errors are artificially injected into this reasoning to form a noisy intermediate result; then, this erroneous reasoning content is fed back to the model, requiring it to continue to perform subsequent reasoning and generate the final answer, thereby training the model to identify, correct or adapt to its own previous errors in subsequent steps and improve its robustness to internal cognitive biases.

[0078] This mechanism artificially alters the intermediate inference results generated by the model itself (for example, deliberately modifying the historical data needed to calculate the growth rate to incorrect values) and feeds back inference content containing logical or factual errors to the model, forcing it to confront its own errors in subsequent inferences. This trains the model to identify, question, and correct internal cognitive biases, ultimately enhancing its robustness in detecting and recovering from its own mistakes in real-world complex tasks.

[0079] In some embodiments, the aforementioned plurality of problem samples may include a second problem sample. Wherein, at least one preset type of noise includes training data noise and / or input noise. Below, this application will use a second problem sample as an example to explain how noise can be injected into the inference process.

[0080] In some embodiments, injecting at least one preset type of noise during the input and / or interaction stages of the intelligent question-answering model processing the second question sample may include the following steps: incorrectly modifying or replacing the annotation information associated with the second question sample; and / or inserting irrelevant, contradictory, or ambiguous interference information into the original content of the second question sample.

[0081] The above steps simulate noise by actively polluting the original question samples or their labeled answers during the training phase. On the one hand, the standard answers corresponding to the questions may be modified or replaced (e.g., replacing the correct answer with an incorrect option) to simulate noise in the training data labeling. On the other hand, irrelevant descriptions, contradictory instructions, or ambiguous expressions may be inserted into the question text to simulate real-world scenarios such as unclear user input and interference from multiple sources of information. This dual-injection mechanism aims to force the intelligent question-answering model to learn under unreliable input and misleading supervision signals, thereby enhancing its ability to identify and clean up input noise and resist interference from mislabeled information, and improving the model's adaptability to low-quality information sources in practical applications.

[0082] Step S206: Using the teacher model, analyze whether the intelligent question answering model identifies and corrects pre-injected noise in each round of question answering training, and determine the noise robustness reward based on whether the intelligent question answering model accurately identifies noise and takes the correct measures; wherein, the noise robustness reward is used to jointly execute the reinforcement learning gradient optimization strategy in conjunction with the task completion reward to update the parameters of the intelligent question answering model.

[0083] In some embodiments, a more powerful teacher model can be used as a "referee" to evaluate the behavior of the intelligent question-answering model in each round of training. The teacher model analyzes whether noise was injected into the training process, whether the model identified the noise, and whether effective corrective measures were taken (such as re-calling tools or correcting erroneous data). Based on this, a quantified "noise robustness reward" is given (e.g., a positive reward for correctly identifying and handling noise, and a negative reward for failing to identify or mishandling it). This specific reward is combined with the task completion quality reward to form the total reward signal for training. Subsequently, the model parameters are optimized through reinforcement learning algorithms (such as PPO), thereby driving the model to actively learn the ability to identify and cope with various types of noise while pursuing task success.

[0084] In some embodiments, the teacher model may refer to a pre-trained language model that has the ability to understand complex instructions and has a larger number of parameters than the model to be trained, or a logical discriminator built based on preset evaluation rules. In one embodiment, the teacher model may specifically be GPT-4 (Generative Pre-trained Transformer 4).

[0085] The noise adaptation method for intelligent question answering models provided in this disclosure systematically defines and progressively injects various preset noises (including tool, model, input, and training data noise) during multiple rounds of training to simulate various interferences and errors in the real environment. At the same time, combined with a noise robustness reward mechanism based on teacher model analysis, the model is guided to actively identify, correct, and adapt to these noises while learning the task objectives. This significantly enhances the robustness, adaptability, and generalization of the model in complex and dynamic real-world application scenarios, effectively overcoming the fundamental defects of traditional methods that rely on manual rules and post-processing schemes, such as high maintenance costs, poor real-time performance, and lack of endogenous adaptability.

[0086] Figure 3 This is a flowchart illustrating a noise injection method according to an exemplary embodiment.

[0087] refer to Figure 3 Injecting at least one preset type of noise into the input and / or interaction stages of an intelligent question-answering model in a progressively increasing manner may include the following steps.

[0088] Step S302: Determine the current training stage based on the current training round of the intelligent question answering model.

[0089] Step S304: Obtain the noise injection probability corresponding to the current training stage according to the predefined progressively increasing noise injection strategy; wherein, the progressively increasing noise injection strategy is configured to: gradually increase the noise injection probability as training progresses until the target probability value is reached.

[0090] The above steps, based on a preset strategy, map the current training stage to a specific noise injection probability value. The core of this strategy lies in dynamically adjusting the training difficulty: setting a low initial injection probability in the early stages of training, and gradually increasing this probability according to a predetermined function rule (such as linear growth, Sigmoid curve growth, or step-like leap) as the number of training rounds (or steps) increases, until a preset upper limit probability value (e.g., 40%) simulating a real noise environment is reached. This allows the model to smoothly transition from "low-noise learning" to "high-noise adaptation," achieving robust and systematic training.

[0091] Step S306: Based on the noise injection probability corresponding to the current training stage, inject at least one preset type of noise into the input and / or interaction stages of the intelligent question answering model.

[0092] In some embodiments, probability sampling can be used to determine whether to inject noise in the current training round. After determining whether to inject noise, one or more preset noise types (such as tool noise, model noise, etc.) can be selected according to the strategy and specifically implanted into the model's input data stream or the model's interaction response with external tools / environment. This simulates interference in the real environment under controllable randomness and drives the model to learn stable reasoning and decision-making capabilities under noisy conditions.

[0093] In some embodiments, according to a preset stepping pattern, the proportion or intensity of injected noise is increased in stages to inject at least one preset type of noise into the data processed by the intelligent question answering model. This may include the following steps: obtaining the current training round of the intelligent question answering model; calculating the noise injection probability corresponding to the current training round according to the ratio of the current training round to the preset total training rounds, based on a preset functional relationship; wherein the preset functional relationship includes a linear function, an sigmoid function, or a step function that monotonically increases with the ratio; and injecting at least one preset type of noise into the data processed by the intelligent question answering model based on the noise injection probability corresponding to the current training round.

[0094] The aforementioned progressive noise injection method achieves a smooth increase in training difficulty by dynamically adjusting the probability of noise occurrence during the training phase: simple noise is introduced with a low probability in the early stage of training to allow the model to solidify its basic task capabilities; as training progresses, the probability and complexity of noise are gradually increased to guide the model to gradually adapt to interference that is closer to the real environment; finally, training is completed under stable noise with a high probability, enabling the model to generate strong noise recognition, filtering and fault tolerance capabilities, thereby significantly improving its robustness, adaptability and generalization performance in complex real-world scenarios.

[0095] Figure 4 This is a flowchart illustrating a noise injection method according to an exemplary embodiment.

[0096] refer to Figure 4 As training progresses, the probability of noise injection is gradually increased until the target probability value is reached. This may include the following steps.

[0097] Step S402: In the early training stage, the noise injection probability is controlled to be within the first probability interval, and is increased linearly or by a step function as the training progresses.

[0098] In some embodiments, in the early stages of training, the probability of noise injection is limited to a low preset range (e.g., 0% to 5%), and the probability is gradually increased as the number of training steps or rounds progresses according to a function rule of linear uniform growth or stepwise segmented leaps. This introduces noise in a gentle and controllable manner in the early stages of model learning, laying a stable foundation for its subsequent adaptation to more complex interferences.

[0099] Step S404: During the intermediate training phase, the noise injection probability is controlled to be within the second probability interval, and is increased linearly or by a step function as the training progresses. The initial value of the second probability interval is not lower than the termination value of the first probability interval.

[0100] In some embodiments, during the mid-stage of training, the probability of noise injection is increased to a higher preset range (e.g., 5% to 25%), and continues to be gradually increased with the training process using a linear or step function. At the same time, it is ensured that the initial probability of this range is not lower than the probability value at the end of the previous stage, so as to ensure that the training difficulty continues to increase rather than decrease, thereby guiding the model to systematically adapt to the noise environment with gradually increasing intensity and complexity, provided that it already has basic capabilities.

[0101] Step S406: In the later training phase, the noise injection probability is stably maintained at the target probability value used to simulate a real noise environment, wherein the termination value of the second probability interval does not exceed the target probability value.

[0102] In some embodiments, in the later stages of training, the noise probability is no longer increased, but is instead stabilized at a pre-set target probability value (e.g., 40%) used to simulate real-world noise levels. At the same time, the probability cap at the end of the previous stage (mid-stage) is required not to exceed this target value. This allows the model to be consolidated in a constant noise environment with near-real-world complexity, ultimately achieving robustness that enables it to work stably in real-world applications.

[0103] In some embodiments, calculating the noise injection probability corresponding to the current training round based on the ratio of the current training round to the preset total training rounds according to a preset functional relationship may include the following steps: if the ratio is less than a first threshold (early stage), controlling the noise injection probability to be within a first probability interval, and increasing it linearly or stepwise with the training progress; if the ratio is greater than or equal to the first threshold and less than a second threshold (mid-stage), controlling the noise injection probability to be within a second probability interval, and increasing it linearly or stepwise with the training progress; wherein the starting value of the second probability interval is not lower than the ending value of the first probability interval; if the ratio is greater than or equal to the second threshold (late stage), maintaining the noise injection probability at the target probability value used to simulate a real noise environment, wherein the ending value of the second probability interval does not exceed the target probability value; wherein the first threshold is greater than the second threshold.

[0104] In general, a learning strategy can be adopted, gradually increasing the proportion of various noise levels. Training can be divided into three stages: 0%-20% training steps: The noise probability increases from 0% to 5%, and the relationship between the noise probability and the number of training steps is linear or a sigmoid function. This stage introduces less noise, aiming to allow the model to learn basic task processing strategies and avoid early training failures; 20%-40% training steps: The noise probability increases from 5% to 25%, and the relationship between the noise probability and the number of training steps is linear, a sigmoid function, or a step function. This stage introduces more noise, allowing the model to gradually adapt to more noise; 40%-100% training steps: The noise probability is maintained at 40%, allowing the model to learn to complete user requests in noisy environments.

[0105] The above method introduces noise progressively, allowing the large model to gradually adapt to complex scenarios and making its training more stable. Noise is introduced progressively using linear functions, sigmoid functions, and step functions. This method avoids situations where the model cannot learn due to high noise levels in the early stages of training, thus improving training stability and model robustness.

[0106] This technical solution significantly enhances the training effect and practical application performance of intelligent question-answering models through a phased and progressively increasing noise injection probability learning strategy. Specifically, the method first introduces noise with low probability in the early stage of training to help the model solidify its basic task capabilities; then, in the middle stage, it systematically increases the noise intensity and complexity, guiding the model to continuously adapt to more severe interference environments; finally, in the later stage, it stabilizes the noise level at a high probability value simulating real-world scenarios, enabling the model to solidify its robustness under high noise conditions. This process effectively avoids learning collapse caused by excessive noise in the early stages of training, ensuring the stability of the training process and ultimately enabling the model to acquire strong intrinsic noise resistance. This allows it to maintain high accuracy and reliability in complex, multi-interference environments in the real world, significantly improving the practical value and deployment adaptability of intelligent question-answering systems.

[0107] Figure 5 This is a flowchart illustrating a noise injection method according to an exemplary embodiment.

[0108] refer to Figure 5 Based on the noise injection probability corresponding to the current training phase, injecting at least one preset type of noise into the input and / or interaction phases of the intelligent question answering model may include the following steps.

[0109] Step S502: Determine whether to perform noise injection in the current training round based on the noise injection probability.

[0110] In some embodiments, the system can determine whether to actually introduce noise in the current round by random sampling based on the noise injection probability (e.g., 20%) corresponding to the current training phase: the system generates a random number, and if the number is less than or equal to the set probability threshold, the noise injection process is triggered; otherwise, no noise is injected, thereby ensuring that the frequency of noise occurrence conforms to the preset progressive course arrangement as a whole, while introducing uncertainty at the level of individual training samples to simulate the randomness of noise occurrence in the real environment.

[0111] Figure 6 This is a flowchart illustrating a noise injection method according to an exemplary embodiment.

[0112] refer to Figure 6 Based on the noise injection probability corresponding to the current training phase, injecting at least one preset type of noise into the input and / or interaction stages of the intelligent question answering model may include the following steps.

[0113] Step S602: Based on the noise injection probability, determine at least one noise type as the target noise type from at least one preset type, and inject noise of at least one target noise type into the input and / or interaction stages of the intelligent question answering model.

[0114] In some embodiments, one or more noise types can be selected from a variety of preset noise types (such as tool noise, model noise, input noise, etc.) according to specific rules (such as equal probability random, type weight, or round alternation) as the target type for this injection, and then specifically implanted into the model input or the interaction with the external system, thereby achieving a diversified combination of noise types and intensities, and more comprehensively simulating complex interference patterns in real-world scenarios.

[0115] The aforementioned noise injection method, through a probability-driven decision-making mechanism, first controls the occurrence of noise in a single training round by random sampling, ensuring the consistency of the overall noise frequency with the course plan and simulating the randomness of real interference. Subsequently, when determining the injection, it further flexibly selects target noise from a variety of preset types for targeted implantation according to the strategy, realizing a diverse combination of noise type, occurrence timing, and introduction method. This constructs a highly realistic and complexity-controllable noise environment during training, effectively improving the model's adaptability and robustness to various unstable factors and complex interferences in real-world scenarios.

[0116] Figure 7 This is a flowchart illustrating a noise robustness reward determination method according to an exemplary embodiment.

[0117] In some embodiments, multi-round intelligent question answering training may include target intelligent question answering training.

[0118] refer to Figure 7 The analysis of the question-answering behavior of the intelligent question-answering model during the training process of the target intelligent question answering model through the teacher model, and the determination of the noise robustness reward corresponding to the training process of the target intelligent question answering model based on the analysis results, may include the following steps.

[0119] Step S702: Analyze the question-answering behavior of the intelligent question-answering model during the training process of the target intelligent question answering model through the teacher model.

[0120] Step S704: If noise is injected during the training of the target intelligent question answering and is correctly identified by the intelligent question answering model, then the noise robustness reward is set to the first value.

[0121] Step S706: If no noise is injected during the training of the target intelligent question answering and the intelligent question answering model does not identify the noise, then the noise robustness reward is set to the second value.

[0122] Step S708: If noise is injected during the training of the target intelligent question answering model but is not correctly identified, the noise robustness reward is set to a third value, which is less than the first value and less than the second value.

[0123] Step S710: If no noise is injected during the training of the target intelligent question answering but is identified as noisy by the intelligent question answering model, then the noise robustness reward is set to the fourth value, which is less than the first value and less than the second value.

[0124] In some embodiments, a noise robustness reward can be introduced into the reinforcement learning reward function. The teacher model (such as the GPT-4o API) is used to analyze the model behavior and calculate the reward. The specific rules are as follows: If the model identifies tool errors, model noise, or data noise and takes remedial measures (such as fixing the format or re-calling the tool): reward +1.0; If no noise is detected: reward 0.0; If noise exists but the model does not identify it: reward -1.0.

[0125] In some embodiments, a reward mechanism can be used to incentivize large models to proactively identify and correct external errors, rather than passively relying on manual verification.

[0126] The above methods can significantly improve the adaptability of large models in real-world environments and reduce reliance on post-processing mechanisms.

[0127] In some embodiments, a large model can be optimized using, for example, a reinforcement learning algorithm, based on the total reward (including task completion reward and noise robustness reward). The training process is repeated, gradually increasing the noise ratio until the large model can stably handle tool instability in real-world environments.

[0128] Through the above iterative training, the large model can maintain efficient and accurate analytical capabilities in complex and ever-changing real-world environments.

[0129] The aforementioned noise robustness reward determination mechanism uses a teacher model to conduct a refined evaluation of the training behavior of the intelligent question answering model, and assigns differentiated reward values ​​based on four combinations of "noise presence" and "model recognition" to accurately guide model learning.

[0130] Specifically: when noise is actually injected during training and the model successfully identifies and processes it, the highest positive reward (first value) is given to incentivize its proactive error correction ability; if the training itself is noise-free and the model does not misclassify, a neutral or basic positive reward (second value) is given to encourage it to maintain a normal state. Conversely, when there is noise but the model fails to identify it, a lower negative reward (third value) is given to punish its missed detection; and when there is no noise but the model misclassifies it as a miss, a negative reward is also given (fourth value) to suppress its oversensitivity and false alarm tendency.

[0131] This mechanism transforms the noise adaptation problem into an optimizable objective by establishing a reinforcement signal that "rewards recognition and processing, and punishes missed detections and false alarms," ​​thereby driving the model to actively and accurately develop robust judgment and response capabilities to various types of noise during training.

[0132] Below, this application will explain and illustrate the noise adaptation method for intelligent question answering models in conjunction with specific application scenarios.

[0133] In some embodiments, the noise adaptation method described above for intelligent question answering models can be deployed in an agent.

[0134] In some embodiments, the above-described intelligent question-answering model can be a large model, but this application is not limited thereto.

[0135] The technical solution proposed in this embodiment will be described in detail below.

[0136] The noise adaptation method described above for intelligent question answering models may include the following steps.

[0137] Step 1: Initialize the training environment.

[0138] In some embodiments, a training environment for a large model can be constructed, considering algorithms such as GRPO (Group Relative Policy Optimization), DAPO (Dynamic Sampling Policy Optimization), DPO (Direct Preference Optimization), and SFT (Supervised Fine-Tuning). Initialization tools may be used; the model may invoke data extraction tools, search tools, browser tools, Python tools, etc.

[0139] Step 2: Noise type definition.

[0140] In some embodiments, large models may encounter a variety of errors, including...

[0141] 1. Tool noise, divided into three categories: Formatting noise: such as incorrect Markdown table formatting returned by the tool, missing column names, changes in null value symbols, and disordered webpage formatting returned by the browser tool.

[0142] Numerical noise: For example, the data returned by the data extraction tool does not match the request, the time range is inconsistent with the query request, important indicators are omitted, indicator definitions are confused, and the content returned by the search tool includes irrelevant web pages, etc.

[0143] Tool call failed: Errors such as tool call failure due to communication delays or other reasons.

[0144] 2. Model noise: The model itself may also generate noise, making some incorrect statements in the preceding reasoning steps, which may affect the subsequent reasoning steps.

[0145] 3. Input noise: User input may contain noise, such as irrelevant information, contradictory instructions, unclear meaning, etc.

[0146] 4. Training data noise: For example, when using DPO or SFT algorithms for training, the training data may contain errors, such as incorrect preference labeling or illusion errors in the response data.

[0147] The above steps simulate common error patterns in the real training and model application stages by precisely defining various noise types.

[0148] The above method enables large models to learn to handle common noise in real-world environments in a targeted manner, thereby improving generalization ability.

[0149] Step 3: Introduce course learning strategies

[0150] In some embodiments, a course-based learning strategy can be employed to gradually increase the proportion of various noises. The training is divided into three phases.

[0151] Phase 1: 0%-20% Training Steps: The noise probability increases from 0% to 5%, and the relationship between the noise probability and the number of training steps is linear or a sigmoid function. This phase has relatively little noise, aiming to allow the model to learn basic task processing strategies first and avoid early training failure.

[0152] Phase Two: 20%-40% Training Steps: The noise probability increases from 5% to 25%. The relationship between the noise probability and the number of training steps is linear, a sigmoid function, or a step function. This phase introduces more noise, allowing the model to gradually adapt to the increased noise levels.

[0153] Phase 3: 40%-100% Training Step: The noise probability is kept at 40%, allowing the model to learn to complete user requests in noisy environments.

[0154] The above process introduces noise gradually, allowing the large model to adapt to complex scenarios and making the training of the large model more stable; noise is introduced gradually using linear functions, sigmoid functions, step functions, etc.

[0155] The above method avoids the situation where the model cannot learn due to high noise in the early stage of training, and improves training stability and model robustness.

[0156] Step 4: Noise robustness reward mechanism.

[0157] In some embodiments, a noise-robust reward can be introduced into the reinforcement learning reward function, and the model behavior can be analyzed using GPT-4oAPI to calculate the reward, with the specific rules as follows.

[0158] 1. Errors in model recognition tools, model noise, or data noise, and remedial measures are taken (such as formatting or re-invoking the tool): Bonus +1.0.

[0159] 2. No noise detected: Bonus 0.0.

[0160] 3. Noise exists but the model does not recognize it: Reward -1.0.

[0161] The above method incentivizes large models to proactively identify and correct external errors through a reward mechanism, rather than passively relying on manual verification.

[0162] The above methods significantly improve the adaptability of large models in real-world environments and reduce reliance on post-processing mechanisms.

[0163] Step 5: Strategy optimization and iteration.

[0164] In some embodiments, the policy of a large model can be optimized using, for example, a reinforcement learning algorithm, based on the total reward (including task completion reward and noise robustness reward). The training process is repeated, gradually increasing the noise ratio until the large model can stably handle tool instability in real-world environments.

[0165] Through iterative training, the above steps enable large models to maintain efficient and accurate analytical capabilities in complex and ever-changing real-world environments.

[0166] The above method has at least the following technical effects.

[0167] 1. Improved robustness: Large models can maintain analytical accuracy even when tools fail to call, return errors, or have data noise.

[0168] 2. Enhanced Adaptability: Without the need for manual rules, the model can proactively identify and correct various errors.

[0169] 3. Generalization ability: The course learning strategy enables the model to adapt to multiple error types.

[0170] 4. Real-time performance: By directly processing noise during the inference stage, the delay of the post-processing mechanism is reduced.

[0171] Figure 8 This is a flowchart illustrating a noise adaptation method for an intelligent question-answering model according to an exemplary embodiment.

[0172] Step S802: Input the training data into the model for inference and introduce progressive noise.

[0173] The progressive noise in step S802 may include model noise, input noise, or training noise.

[0174] Step S804: Call the tool based on the model output and introduce progressive noise during the call.

[0175] The progressive noise in step S804 may include tool noise, etc.

[0176] Step S806: Continue reasoning until the reasoning is completed.

[0177] Step S808: Calculate the reward based on the complete output of the model and optimize the strategy model.

[0178] Step S810, determine if training is complete?

[0179] In some embodiments, if it is determined that the training has not yet ended, steps S802 to S810 are executed repeatedly.

[0180] This technical solution systematically introduces progressive noise throughout the entire model inference process (including data input, model inference, external tool calls, and final result generation), constructing a dynamic, noise-resistant training environment covering the entire "input-processing-output" process. This end-to-end noise exposure mechanism enables the intelligent question-answering model to learn how to remain stable and complete tasks at different stages (such as handling noisy input, correcting its own inference biases, and adapting to unreliable tool feedback) under near-real-world complexity interference. Furthermore, it uses the comprehensive reward of the final output to optimize its strategy, thereby iteratively cultivating an intrinsic and robust noise robustness and environmental adaptability throughout the entire question-answering process.

[0181] It should be particularly noted that the steps in the various embodiments of the noise adaptation method for intelligent question answering models described above can be overlapped, substituted, added, or deleted from each other. Therefore, these reasonable permutations and combinations of the noise adaptation method for intelligent question answering models should also fall within the protection scope of this disclosure, and the protection scope of this disclosure should not be limited to the described embodiments.

[0182] It should be noted that the scope of protection of this application should include, but is not limited to, the specific implementation methods described in the embodiments. Any alternative solution that uses a different name but substantially performs the same function and achieves the same technical effect falls within the scope of protection defined by the claims of this application.

[0183] Based on the same inventive concept, this disclosure also provides a noise adaptation device for an intelligent question-answering model, as described in the following embodiments. Since the principle of this device embodiment in solving the problem is similar to that of the above-described method embodiments, the implementation of this device embodiment can refer to the implementation of the above-described method embodiments, and repeated details will not be elaborated further.

[0184] Figure 9 This is a block diagram illustrating a noise adaptation device for an intelligent question-answering model according to an exemplary embodiment. (Refer to...) Figure 9 The noise adaptation device 900 for intelligent question answering models provided in this embodiment may include: a sample acquisition module 901, a noise injection module 902, and a noise response capability analysis module 903.

[0185] The sample acquisition module 901 can be used to acquire an intelligent question-answering model and multiple question samples; the noise injection module 902 can be used to perform multiple rounds of intelligent question-answering training on the intelligent question-answering model based on the multiple question samples; during the multiple rounds of intelligent question-answering training, according to a preset stepping rule, the proportion or intensity of injected noise is increased in stages, injecting at least one preset type of noise into the data processed by the intelligent question-answering model; the noise response capability analysis module 903 can be used to analyze whether the intelligent question-answering model identifies and corrects the pre-injected noise in each round of question-answering training through a teacher model, and determine the noise robustness reward based on whether the intelligent question-answering model accurately identifies the noise and takes the correct measures; wherein, the noise robustness reward is used to combine the task completion reward with the joint execution of the reinforcement learning gradient optimization strategy to drive the parameter update of the intelligent question-answering model.

[0186] It should be noted that the sample acquisition module 901, noise injection module 902, and noise response capability analysis module 903 mentioned above correspond to S202 to S406 in the method embodiment. The examples and application scenarios implemented by the above modules and their corresponding steps are the same, but they are not limited to the content disclosed in the above method embodiment. It should be noted that the above modules, as part of the device, can be executed in a computer system such as a set of computer-executable instructions.

[0187] In some embodiments, according to a preset stepping pattern, the proportion or intensity of injected noise is increased in stages to inject at least one preset type of noise into the data processed by the intelligent question-answering model, including: obtaining the current training round of the intelligent question-answering model; calculating the noise injection probability corresponding to the current training round according to the ratio of the current training round to the preset total training rounds, based on a preset functional relationship; wherein, the preset functional relationship includes a linear function, an sigmoid function, or a step function that monotonically increases with the ratio; and injecting at least one preset type of noise into the data processed by the intelligent question-answering model based on the noise injection probability corresponding to the current training round.

[0188] In some embodiments, the noise injection probability corresponding to the current training round is calculated according to a preset functional relationship based on the ratio of the current training round to the preset total training rounds, including: if the ratio is less than a first threshold, controlling the noise injection probability to be in a first probability interval, and increasing it linearly or stepwise with the training progress; if the ratio is greater than or equal to the first threshold and less than a second threshold, controlling the noise injection probability to be in a second probability interval, and increasing it linearly or stepwise with the training progress; wherein the starting value of the second probability interval is not lower than the ending value of the first probability interval; if the ratio is greater than or equal to the second threshold, maintaining the noise injection probability at a target probability value used to simulate a real noise environment, wherein the ending value of the second probability interval does not exceed the target probability value; wherein the first threshold is greater than the second threshold.

[0189] In some embodiments, according to a preset stepping pattern, the proportion or intensity of injected noise is increased in stages to inject at least one preset type of noise into the data processed by the intelligent question answering model, including: determining whether to perform noise injection in the current training round based on the noise injection probability; and / or, based on the noise injection probability, determining at least one noise type as a target noise type among the at least one preset type, and injecting at least one preset type of noise into the data processed by the intelligent question answering model.

[0190] In some embodiments, the at least one preset type of noise includes at least one of tool noise, model noise, input noise, and training data noise; the tool noise originates from errors generated when the intelligent question answering model calls external tools; the model noise originates from errors generated by the intelligent question answering model's own reasoning; the input noise originates from errors carried in the question samples; and the training data noise originates from errors in the labeled data used to train the intelligent question answering model.

[0191] In some embodiments, the plurality of question samples includes a first question sample or a second question sample; wherein, injecting at least one preset type of noise into the data processed by the intelligent question answering model includes at least one of the following methods: when the intelligent question answering model requests data from an external tool, returning the tool noise to the intelligent question answering model, the tool noise including at least one of format error noise, numerical error noise, or tool call failure noise; obtaining the pre-reasoning generated by the intelligent question answering model for the first question sample; injecting errors into the pre-reasoning to obtain erroneous pre-reasoning; sending the erroneous pre-reasoning back to the intelligent question answering model so that the intelligent question answering model continues to perform subsequent reasoning for the first question sample based on the erroneous pre-reasoning and finally outputs the answer to the first question sample; modifying or replacing the annotation information associated with the second question sample; inserting irrelevant, contradictory, or ambiguous interference information into the original content of the second question sample.

[0192] In some embodiments, the multi-round intelligent question-answering training includes target intelligent question-answering training; wherein, through a teacher model, the system analyzes whether the intelligent question-answering model identifies and corrects pre-injected noise in each round of question-answering training, and determines a noise robustness reward based on whether the intelligent question-answering model accurately identifies noise and takes correct measures, including: analyzing the question-answering behavior of the intelligent question-answering model during the target intelligent question-answering training process through the teacher model; if noise is injected during the target intelligent question-answering training process and is identified by the intelligent question-answering model, then the noise robustness reward is set to a first value; if... If no noise is injected during the training of the target intelligent question answering system and the intelligent question answering model does not identify the noise, then the noise robustness reward is set to a second value. If noise is injected during the training of the target intelligent question answering system but is not identified by the intelligent question answering model, then the noise robustness reward is set to a third value, which is less than the first value and less than the second value. If no noise is injected during the training of the target intelligent question answering system but is identified as having noise by the intelligent question answering model, then the noise robustness reward is set to a fourth value, which is less than the first value and less than the second value.

[0193] Since the functions of the device 900 have been described in detail in their respective method embodiments, they will not be repeated here.

[0194] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a portion of a module or program segment containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer program instructions.

[0195] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0196] Figure 10 A schematic diagram of an electronic device suitable for implementing embodiments of the present disclosure is shown. It should be noted that... Figure 10 The illustrated electronic device 1000 is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0197] like Figure 10 As shown, the electronic device 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage section 1008 into a random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the electronic device 1000. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0198] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1010 as needed so that computer programs read from it can be installed into storage section 1008 as needed.

[0199] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing computer program instructions for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs the functions defined above in the system of this disclosure.

[0200] It should be noted that the computer-readable storage medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable computer program instructions. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Computer program instructions contained on a computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0201] In another aspect, this disclosure also provides a computer-readable storage medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable storage medium carries one or more programs that, when executed by the device, enable the device to perform the following functions: acquiring an intelligent question-answering model and multiple question samples; performing multiple rounds of intelligent question-answering training on the intelligent question-answering model based on the multiple question samples; injecting at least one preset type of noise into the input and / or interaction stages of the intelligent question-answering model in a progressively increasing manner during the multiple rounds of intelligent question-answering training; analyzing the question-answering behavior of the intelligent question-answering model in each round of intelligent question-answering training using a teacher model, and determining the noise robustness reward corresponding to each round of intelligent question-answering training based on the analysis results; wherein the noise robustness reward is used to quantify the intelligent question-answering model's ability to identify and process noise; wherein the noise robustness reward is used to combine with task completion rewards to jointly execute a reinforcement learning gradient optimization strategy to drive parameter updates of the intelligent question-answering model.

[0202] According to one aspect of this disclosure, a computer program product or computer program is provided, comprising computer program instructions stored in a computer-readable storage medium. The computer program instructions are read from the computer-readable storage medium, and a processor executes the computer program instructions to implement the methods provided in various optional implementations of the above embodiments.

[0203] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions of the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive) and includes several computer program instructions to cause an electronic device (such as a server or terminal device) to execute the method according to the embodiments of this disclosure.

[0204] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0205] It should be understood that this disclosure is not limited to the detailed structures, drawing arrangements or implementation methods shown herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A noise adaptive method for intelligent question answering models, characterized in that, include: Obtain the intelligent question-answering model and multiple question samples; Based on the multiple question samples, the intelligent question answering model is trained in multiple rounds. During the multi-round intelligent question answering training process, according to the preset stepping rules, the proportion or intensity of injected noise is increased in stages, and at least one preset type of noise is injected into the data processed by the intelligent question answering model. Using a teacher model, we analyze whether the intelligent question-answering model identifies and corrects pre-injected noise in each round of question-answering training, and determine the noise robustness reward based on whether the intelligent question-answering model accurately identifies noise and takes the correct measures. The noise robustness reward is used in conjunction with the task completion reward to jointly execute a reinforcement learning gradient optimization strategy to update the parameters of the intelligent question answering model.

2. The method according to claim 1, characterized in that, According to a preset stepping pattern, the proportion or intensity of injected noise is increased in stages, injecting at least one preset type of noise into the data processed by the intelligent question-answering model, including: Obtain the current training round of the intelligent question-answering model; Based on the ratio of the current training round to the preset total training rounds, the noise injection probability corresponding to the current training round is calculated according to a preset functional relationship; wherein, the preset functional relationship includes a linear function, a sigmoid function, or a step function that monotonically increases with the ratio. Based on the noise injection probability corresponding to the current training round, at least one preset type of noise is injected into the data processed by the intelligent question answering model.

3. The method according to claim 2, characterized in that, Based on the ratio of the current training round to the preset total number of training rounds, the noise injection probability corresponding to the current training round is calculated according to a preset functional relationship, including: If the ratio is less than the first threshold, the noise injection probability is controlled to be within the first probability interval, and is increased linearly or by a step function as the training progress progresses. If the ratio is greater than or equal to the first threshold and less than the second threshold, the noise injection probability is controlled to be in the second probability interval, and is increased linearly or stepwise as the training progress progresses; wherein, the starting value of the second probability interval is not lower than the ending value of the first probability interval; If the ratio is greater than or equal to the second threshold, the noise injection probability is maintained at the target probability value used to simulate a real noise environment, wherein the termination value of the second probability interval does not exceed the target probability value; wherein the first threshold is greater than the second threshold.

4. The method according to claim 2, characterized in that, According to a preset stepping pattern, the proportion or intensity of injected noise is increased in stages, injecting at least one preset type of noise into the data processed by the intelligent question-answering model, including: Whether to perform noise injection in the current training round is determined based on the noise injection probability. And / or, based on the noise injection probability, determine at least one noise type as a target noise type from the at least one preset type, and inject at least one preset type of noise into the data processed by the intelligent question answering model.

5. The method according to claim 1, characterized in that, The at least one preset type of noise includes at least one of tool noise, model noise, input noise, and training data noise; The tool noise originates from errors generated when the intelligent question-answering model calls external tools. The model noise originates from errors generated by the intelligent question-answering model's own reasoning. The input noise originates from errors carried in the problem samples. The noise in the training data stems from errors in the labeled data used to train the intelligent question-answering model.

6. The method according to claim 5, characterized in that, The plurality of question samples includes a first question sample or a second question sample; wherein, injecting at least one preset type of noise into the data processed by the intelligent question-answering model includes at least one of the following methods: When the intelligent question-answering model requests data from an external tool, the tool noise is returned to the intelligent question-answering model. The tool noise includes at least one of format error noise, numerical error noise, or tool call failure noise. Obtain the preceding reasoning generated by the intelligent question answering model for the first question sample; inject errors into the preceding reasoning to obtain preceding reasoning containing errors; feed the preceding reasoning containing errors back to the intelligent question answering model so that the intelligent question answering model can continue to perform subsequent reasoning for the first question sample based on the preceding reasoning containing errors, and finally output the answer to the first question sample. The annotation information associated with the second problem sample was incorrectly modified or replaced; Irrelevant, contradictory, or ambiguous distracting information is inserted into the original content of the second problem sample.

7. The method according to claim 1, characterized in that, The multi-round intelligent question-answering training includes target intelligent question-answering training; wherein, through a teacher model, the training analyzes whether the intelligent question-answering model identifies and corrects pre-injected noise in each round of question-answering training, and determines noise robustness rewards based on whether the intelligent question-answering model accurately identifies noise and takes correct measures, including: The teacher model is used to analyze the question-answering behavior of the intelligent question-answering model during the training process of the target intelligent question answering system. If noise is injected during the training of the target intelligent question answering and is recognized by the intelligent question answering model, then the noise robustness reward is set to the first value. If no noise is injected during the training of the target intelligent question answering and the intelligent question answering model does not identify the noise, then the noise robustness reward is set to the second value. If noise is injected during the training of the target intelligent question answering system but is not recognized by the intelligent question answering model, then the noise robustness reward is set to a third value, which is less than the first value and less than the second value. If no noise is injected during the training of the target intelligent question answering, but the intelligent question answering model identifies it as noisy, then the noise robustness reward is set to a fourth value, which is less than the first value and less than the second value.

8. An electronic device, characterized in that, include: Memory and processor; The memory is used to store computer program instructions; the processor calls the computer program instructions stored in the memory to implement the noise adaptation method for intelligent question answering models as described in any one of claims 1-7.

9. An electronic device, characterized in that, include: Memory and processor; The memory is used to store computer program instructions; the processor calls the computer program instructions stored in the memory to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method as described in any one of claims 1-7.

Images