Intelligent question-answering system reinforcement learning method and system based on multistage evaluation feedback

Abstract

The invention discloses an intelligent question-answering system reinforcement learning optimization method and system based on multistage evaluation feedback, and belongs to the technical field of intelligent question-answering reinforcement learning, and the method comprises the following steps: S1, constructing an iterative training framework fusing multistage evaluation feedback and reinforcement learning decision; s2, based on the constructed iterative training framework, performing hierarchical evaluation on the intelligent question-answering system to obtain an optimization signal; s3, based on the obtained optimization signal, driving a dynamic adjustment strategy of a reinforcement learning agent; and S4, based on a dynamic adjustment strategy, in combination with risk characteristics of an application scene, implementing differential threshold management, and finally forming a closed-loop performance optimization mechanism. The invention aims to construct an iterative training framework fusing multi-level evaluation feedback and reinforcement learning decision, and the performance of the intelligent question-answering system is continuously improved.

Description

Technical Field

[0001] This application belongs to the field of reinforcement learning technology for intelligent question answering, and in particular relates to methods and systems for reinforcement learning of intelligent question answering systems based on multi-level evaluation feedback. Background Technology

[0002] As one of the core applications in the fields of natural language processing and artificial intelligence, intelligent question-answering systems are widely used in dynamic scenarios such as e-commerce consultation, medical diagnosis, and financial customer service. Their core function is to understand users' natural language questions, accurately match information from massive amounts of data, and generate answers that meet their needs, thus providing users with efficient information interaction services.

[0003] In existing technologies, the core process of intelligent question answering systems is usually a two-step process of "retrieval-generation": the retrieval step relies on preset text segmentation rules to extract relevant information from documents, and the generation step integrates and outputs the retrieved information based on a language model with fixed parameters.

[0004] However, fixed segmentation rules during retrieval can easily lead to fragmentation of key information and damage the integrity of the context; the generation stage lacks accurate comparison with the retrieved information, which can easily result in fabricated or contradictory content; and the lack of a linkage evaluation mechanism between retrieval quality, generation accuracy and user feedback makes it impossible to form a closed-loop optimization link, making it difficult to adapt to the complex needs in dynamic scenarios, thus limiting the improvement of system performance.

[0005] The retrieval process often relies on fixed text segmentation rules or simple keyword matching. For example, the patent application CN201711419805X, filed on December 25, 2017, entitled "Method, Apparatus, Device, and Question-Answer System for Generating Intelligent Question-Answering Systems," does not address the effective handling of fragmented key information during the retrieval process. When faced with complex documents, it is difficult to ensure the integrity of contextual information, leading to inaccurate retrieval results and affecting subsequent answer generation.

[0006] Existing technologies lack a multi-level evaluation and feedback mechanism in the generation stage. When generating answers, existing technologies do not effectively integrate and evaluate retrieval quality, generation accuracy, and user feedback. For example, patent CN2025100636509, filed on January 15, 2025, only focuses on how model structure improves the quality of the response text, without considering multi-dimensional evaluation to optimize the entire question-and-answer process. This makes the generated answers prone to contradictions with the retrieved information and fabricated content, failing to meet users' needs for accuracy and reliability.

[0007] Furthermore, existing technologies lack a closed-loop optimization mechanism. Most existing technologies only improve intelligent question-answering systems from a single perspective, such as optimizing the construction process or adjusting the model structure, without forming a closed-loop iterative mechanism from evaluation and feedback to optimization. When the system operating environment changes or user needs change, it cannot adapt and adjust in a timely manner, making it difficult to continuously improve system performance. Summary of the Invention

[0008] To address the aforementioned shortcomings of existing technologies, the objective of this application is to propose a reinforcement learning method for intelligent question-answering systems based on multi-level evaluation feedback. This method aims to construct an iterative training framework that integrates multi-level evaluation feedback and reinforcement learning decision-making, forming a closed-loop mechanism of "evaluation-feedback-optimization." This solves the problem of existing technologies being unable to dynamically adapt to environmental changes and user needs, thereby continuously improving the performance of intelligent question-answering systems. The first aspect of this application provides a reinforcement learning optimization method for intelligent question-answering systems based on multi-level evaluation feedback. This method includes the following steps: S1. Construct an iterative training framework that integrates multi-level evaluation feedback and reinforcement learning decision-making; S2. Based on the constructed iterative training framework, a hierarchical evaluation of the intelligent question answering system is performed to obtain optimization signals; S3. Based on the acquired optimization signals, a dynamic adjustment strategy is used to drive the reinforcement learning agent. S4. Based on the dynamic adjustment strategy and combined with the risk characteristics of the application scenario, implement differentiated threshold management to ultimately form a closed-loop performance optimization mechanism.

[0009] Furthermore, the construction of the iterative training framework in S1 further includes: The evaluation module is used to evaluate each part of the system. The evaluation module has a built-in multi-dimensional evaluation index library, and each index supports dynamic weight configuration. The feedback step involves converting the evaluation results into optimization signals. The signal quantizer maps the qualitative evaluation results into quantized scores, ensuring that the signals can be directly parsed by the reinforcement learning model. The optimization module adjusts the strategy based on the optimized signal drive, and the optimization steps are optimized.

[0010] Furthermore, the optimization module further includes a strategy generator and an execution engine. The strategy generator outputs specific adjustment instructions, and the execution engine interfaces with the API of the retrieval and generation components to implement the strategy.

[0011] Furthermore, the evaluation module, feedback module, and optimization module interact through a real-time data bus to form an iterative closed loop of evaluation data input → feedback signal output → optimization strategy execution → evaluation data update.

[0012] Furthermore, the intelligent question-answering system further includes a retrieval module and a generation module; The retrieval module is used to obtain contextual information related to the question from the document. The generation module generates answers based on the retrieved context information, using a pre-trained language model, and the generation process supports evidence citation annotation. The information retrieval quality of the retrieval module and the answer output quality of the generation module are evaluated hierarchically.

[0013] Furthermore, the hierarchical evaluation of the retrieval module includes the following steps: S2.1 Analyze the structural features of the document, including the heading hierarchy tree and table anchor point positions. The heading hierarchy tree is constructed by analyzing the document outline tags. S2.2. Based on the structural features of the parsed document, dynamically evaluate the rationality of text segmentation, calculate the breakage rate between headings and table anchor points, and calculate the breakage rate R. f The formula is:

[0014] in: The number of times the title and related tables are split into different blocks; : Total number of title-table related pairs in the document; S2.3, when the fracture rate R f When the threshold δ is exceeded, a block reorganization mechanism is automatically triggered to ensure the integrity and accuracy of the context information. The block reorganization mechanism is cross-block semantic aggregation, which calculates the semantic similarity of adjacent blocks and merges blocks with a similarity ≥ 0.85. S2.4 Decompose the generated answer into atomic statements, which is achieved through dependency parsing, breaking down complex sentences into simple statements; S2.5. Cross-compare the obtained atomic statements with the context information acquired by the retrieval module. This cross-compare is achieved by calculating semantic similarity. The atomic statements and context text are converted into vectors, and the matching degree is calculated using the formula: Similarity = (Atomic Statement Vector) / (Atomic Statement Vector) Context vector) / (|| atomic statement vector|| × || context vector||); S2.6. Based on the comparison results, identify and filter fictitious, contradictory, or ambiguous content, and make targeted corrections to the answers for different scenarios; S2.7 Collect behavioral feedback data from end users, including interaction duration, second question rate, and answer acceptance rate; S2.8. Integrate the segmented quality assessment results of the retrieval module, the answer accuracy verification results of the generation module, and the collected user behavior feedback data to form a three-layer optimization signal, which is then input into the decision engine and integrated into a vector [Q]. ret Q gen Q user ], where Q ret For the retrieval quality score: (100×(1-R) f )), Q gen To generate a quality score: (100 × average semantic similarity), Q user User feedback score: (50 × adoption rate + 30 × (1 - second question rate) + 20 × (interaction duration / average duration)).

[0015] Furthermore, the dynamic adjustment strategy of the reinforcement learning agent in S3 further includes the following steps: S3.1 Quantify the obtained multi-level evaluation results and convert them into a reward function for reinforcement learning; The reward function is: R = α·Q ret + β·Q gen + γ·Q user , where α, β, and γ are default weights that can be dynamically adjusted according to the scenario; S3.2 Training the agent to autonomously adjust retrieval and generation strategies based on the reward function; The strategy adjustments include enhancing keyword retrieval weights and enabling multi-round searches, when Q ret When Q < 70, increase the weight of core keywords; when Q gen When the value is less than 65, supplementary search terms are generated for questionable atomic statements, triggering multiple rounds of searches to supplement evidence; S3.3. Accumulate efficient decision-making paths through continuous learning and build a historical decision-making knowledge base to support strategy adjustments.

[0016] A second aspect of this application provides a reinforcement learning optimization system for intelligent question answering systems based on multi-level evaluation feedback, the system comprising: Build modules are used to construct an iterative training framework that integrates multi-level evaluation feedback and reinforcement learning decision-making; The evaluation and optimization module, based on the constructed iterative training framework, performs hierarchical evaluation of the intelligent question answering system to obtain optimization signals; The policy adjustment module, based on the acquired optimization signals, is used to drive the reinforcement learning agent to dynamically adjust its policy. The management module, based on dynamic adjustment strategies and combined with the risk characteristics of application scenarios, implements differentiated threshold management, ultimately forming a closed-loop performance optimization mechanism.

[0017] A third aspect of this application provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the steps of a reinforcement learning method for an intelligent question-answering system based on multi-level evaluation feedback.

[0018] A fourth aspect of this application provides a computer device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement a reinforcement learning method for an intelligent question-answering system based on multi-level evaluation feedback.

[0019] The innovation of this application lies in: (i) Construct an iterative training framework that integrates multi-level evaluation feedback and reinforcement learning decision-making, including an evaluation module, a feedback module and an optimization module, and the three modules are interconnected to form a closed loop; (ii) Conduct a hierarchical evaluation of the retrieval process of the intelligent question-answering system, covering document structure feature analysis, text segmentation rationality evaluation, and segmentation reorganization mechanism; (iii) Conduct hierarchical evaluation during the generation stage, including breaking down the generated answer into atomic statements, cross-referencing them with the retrieval context information, identifying and filtering fictitious, contradictory or ambiguous content, and correcting the answer for different scenarios; (iv) Integrate the segmented quality assessment results of the retrieval process, the accuracy verification results of the answer generation stage, and the behavioral feedback data of end users to form a three-layer optimization signal; (v) Based on the optimization signal-driven dynamic adjustment strategy of reinforcement learning agent, it involves reward function design, specific implementation of strategy adjustment and application of historical decision knowledge base; (vi) Based on the risk characteristics of the application scenarios, implement differentiated threshold management for high-risk areas such as medical care and finance, and efficiency-priority scenarios such as e-commerce consulting.

[0020] The beneficial technical effects that this application can achieve are: 1. By constructing a closed-loop iterative framework of "evaluation-feedback-optimization," multi-level evaluation feedback is integrated with retrieval chunk quality, generation accuracy, and user behavior. Quantitative results are transformed into reinforcement learning reward signals, driving the agent to autonomously adjust its strategy. Combined with an efficient path for reusing historical decision knowledge bases, the system addresses information fragmentation and contradictory answer issues in dynamic scenarios, achieving continuous iterative improvement in system performance.

[0021] 2. By implementing differentiated threshold management based on the risk characteristics of different scenarios, strict review and manual verification mechanisms are set up for high-risk fields such as healthcare and finance, while indicators are relaxed to balance efficiency for scenarios such as e-commerce consultation. Through adaptive threshold adjustment and dynamic framework updates, the system balances accuracy and efficiency, significantly improving the adaptability of the intelligent question-answering system in complex scenarios. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the reinforcement learning method for the intelligent question-answering system based on multi-level evaluation feedback proposed in this application. Detailed Implementation

[0023] To better illustrate the technical solution of this application, this application will be further described in conjunction with the accompanying drawings and specific examples.

[0024] like Figure 1 As shown, this application proposes a reinforcement learning optimization method for intelligent question answering systems based on multi-level evaluation feedback, including the following steps: S1. Construct an iterative training framework that integrates multi-level evaluation feedback and reinforcement learning decision-making; S2. Based on the iterative training framework built by S1, hierarchical evaluation is performed on the retrieval and generation stages of the intelligent question answering system to obtain optimization signals. S3, based on the optimized signal obtained in S2, is used to drive the reinforcement learning agent to dynamically adjust the strategy; S4, based on the dynamic adjustment strategy of S3, combined with the risk characteristics of the application scenario, implements differentiated threshold management, and finally forms a closed-loop performance optimization mechanism.

[0025] In this embodiment, the construction of the iterative training framework in S1 specifically includes: The evaluation module is used to evaluate each aspect of the system. This module has a built-in multi-dimensional evaluation indicator library, covering core indicators such as retrieval completeness, generation accuracy, and user satisfaction, and each indicator supports dynamic weight configuration. The feedback module is used to transform the evaluation results into optimization signals. The signal quantizer maps qualitative evaluation results (such as "reasonable block division" and "contradictory answer") to a quantized score of 0-100, ensuring that the signal can be directly parsed by the reinforcement learning model. The optimization module is used to adjust the strategy based on the optimization signal. It includes a strategy generator and an execution engine. The strategy generator outputs specific adjustment instructions, and the execution engine connects to the API interface of the retrieval and generation components to implement the strategy. The three modules interact through a real-time data bus, forming an iterative closed loop of "evaluation data input → feedback signal output → optimization strategy execution → evaluation data update".

[0026] In this embodiment, the intelligent question-answering system in S2 refers to a system used to implement intelligent question-answering functions in dynamic scenarios. The intelligent question-answering system includes a retrieval stage and a generation stage. The retrieval phase is used to obtain contextual information related to the question from the document. It is implemented through a retrieval engine (such as Elasticsearch) and supports keyword retrieval, semantic retrieval and mixed retrieval modes. The generation phase is used to generate answers based on the retrieved context information. It is implemented using pre-trained language models (such as BART and LLaMA), and the generation process supports evidence citation annotation. Hierarchical evaluation of the retrieval and generation stages refers to assessing the quality of information acquisition in the retrieval stage and the quality of answer output in the generation stage, respectively. Retrieval quality focuses on the completeness and accuracy of information, while generation quality focuses on the accuracy and consistency of answers.

[0027] In this embodiment, S2, the hierarchical evaluation of the retrieval phase includes the following steps: S2.1 Analyze the structural features of the document, including the heading hierarchy tree and table anchor points. The heading hierarchy tree is constructed by parsing the document outline tags (such as "Heading 1 - Heading 3" in Word), forming a four-level structure of "Document - Chapter - Section - Item". The table anchor points are determined by identifying the title text and page number information associated with the table. For example, "Table 3.2 Product Parameter Table" is anchored to the "3.2 Product Specifications" chapter. S2.2. Based on the structural features parsed in S2.1, the rationality of text segmentation is dynamically evaluated by calculating the breakage rate between the title and table anchor points. The formula is: Breakage Rate R f ; Furthermore, the rationality of text segmentation is dynamically evaluated based on the structural features parsed by S2.1, which is achieved by calculating the breakage rate between the title and table anchor points. The formula is as follows:

[0028] in: N broken The number of times the title and related tables are split into different blocks; N total : Total number of title-table related pairs in the document; For example: A medical document contains 20 header-table pairs. If 5 pairs are detected to be segmented, then: .

[0029] S2.3 When S2.2 detects fragmentation of key information, that is, the fragmentation rate R... f When the threshold δ is exceeded, a block reorganization mechanism is automatically triggered to ensure the integrity and accuracy of the context information. The block reorganization mechanism is a cross-block semantic aggregation. Specifically, it calculates the semantic similarity of adjacent blocks through Sentence-BERT and merges blocks with a similarity ≥ 0.85. For broken blocks containing the same entity, they are forcibly merged into complete semantic units. For example, "Material: Cotton" and "Content: 80%" are merged into "Material: Cotton (Content 80%)".

[0030] Furthermore, the block reorganization mechanism is a cross-block semantic aggregation, specifically including: Using the fracture anchor point as the center, select two blocks before and after it as candidate sets; Use Sentence-BERT to calculate the similarity between adjacent blocks within the candidate set; Merge pairs starting with the highest similarity until there are no more pairs with a similarity of ≥0.85. In this embodiment, S2, the hierarchical evaluation of the generation stage includes the following steps: S2.4 Decompose the generated answer into atomic statements, which is achieved through dependency parsing. The Stanford Parser tool is used to extract the subject-verb-object structure and decompose complex sentences into simple statements. For example, "This product is taken twice a day, 5mg each time" is decomposed into two atomic statements: "This product is taken twice a day" and "This product is taken 5mg each time". S2.5. Cross-compare the atomized statements obtained in S2.4 with the context information obtained in the retrieval stage of S2. This cross-compare is achieved by calculating semantic similarity using a cosine similarity algorithm. The atomized statements and context text are converted into vectors, and the matching degree is calculated using the formula: Similarity = (Atomized statement vector) / (Atomized statement vector) Context vector) / (|| atomic statement vector|| × || context vector||); S2.6. Based on the comparison results of S2.5, identify and filter fictitious, contradictory, or ambiguous content, and perform targeted corrections on the answers for different scenarios. The filtering condition is: semantic similarity is lower than the threshold γ. For example, in the medical scenario, if the similarity between the atomic statement "daily dose 3 times" and the context "maximum 2 times per day" is 0.3 < γ = 0.7, then the statement is filtered and corrected to "daily dose 2 times". In the e-commerce scenario, if the similarity between "price 99 yuan" and the retrieved "promotional price 89 yuan" is 0.2 < γ = 0.6, then it is corrected to "current promotional price 89 yuan".

[0031] In this embodiment, S2, obtaining the optimized signal further includes: S2.7 Collect user behavior feedback data, including interaction duration (the duration for which a user views an answer), repeat question rate (the number of times the same question is asked repeatedly / the total number of questions asked), and answer acceptance rate (the number of times a user clicks the "helpful" button / the total number of views). S2.8. Integrate the segmented quality assessment results from the retrieval phase, the answer accuracy verification results from the generation phase, and the user behavior feedback data collected in S2.7 to form a three-layer optimization signal, which is then input into the decision engine. Specifically, this integration is represented by a vector [Q]. ret Qgen Q user ], where Q ret To retrieve the quality score (100×(1-R) f )), Q gen To generate a quality score (100 × average semantic similarity), Q user The user feedback score is calculated as follows: (50 × Adoption Rate + 30 × (1 - Second Question Rate) + 20 × (Interaction Duration / Average Duration)).

[0032] In this embodiment, in S3, the reinforcement learning agent refers to an intelligent entity trained by a reinforcement learning algorithm to autonomously adjust the strategy of the intelligent question answering system. It is trained using the PPO (Proximal Policy Optimization) algorithm, with a state space of three-layer optimization signal vectors and an action space of retrieval strategy adjustment (such as keyword weight and retrieval range) and generation strategy adjustment (such as generation length and evidence citation strength). The dynamic adjustment strategy in S3 specifically includes the following steps: S3.1 Quantify the multi-level evaluation results obtained in S2 into a reward function for reinforcement learning; The reward function is: R = α·Q ret + β·Q gen + γ·Q user The default weights are α=0.3, β=0.4, and γ=0.3, which can be dynamically adjusted according to the scenario. For example, in a medical scenario, β can be increased to 0.5 to enhance the weight of generation accuracy.

[0033] S3.2. Based on the reward function of S3.1, train the agent to autonomously adjust the retrieval and generation strategies; The strategy adjustments include enhancing keyword retrieval weights and enabling multi-round searches, when Q ret When Q < 70, increase the weight of core keywords (such as disease names in medical questions) from 0.5 to 0.8; when Q gen When the value is less than 65, supplementary search terms (such as "maximum dose of aspirin") are generated for questionable atomic statements, triggering multiple rounds of searches to supplement evidence; S3.3. Accumulate efficient decision-making paths through continuous learning and build a historical decision-making knowledge base to support the strategy adjustment in S3.2. For example, in the medical scenario, "[Q ret =70, Q gen =60, Q user =75]→[Call terminology base + multi-round retrieval]→reward gain + 15” is a successful path. When the similarity between the new state and the historical state is ≥0.9, the action sequence is directly reused. The decision path is stored as a triplet consisting of a state feature vector, an action sequence, and a reward gain.

[0034] In this embodiment, the risk characteristics of the application scenarios in S4 refer to the different requirements of different application scenarios for the accuracy, security and efficiency of the answers. High-risk scenarios (medical, financial) prioritize accuracy and security, while efficiency-first scenarios (e-commerce consulting) prioritize balancing efficiency and practicality. Combining the risk characteristics of application scenarios means developing corresponding management strategies based on the risk characteristics of different scenarios. The implementation of differentiated threshold management in S4 specifically includes: S4.1. Strict content review standards are set for high-risk areas such as healthcare and finance; credibility = 0.6 × Q gen +0.4× Terminology database matching degree. The terminology database adopts UMLS (medical) and financial terminology database (issued by the central bank). When necessary, manual review is triggered and pushed to the professional workbench through the workflow system. The answer is only output after the review is approved. When the generated answer's credibility is less than 0.7, the terminology database will be automatically invoked for verification. When the credibility of the generated answer is less than 0.5, manual review will be initiated. Furthermore, considering that when the answer contains terms from a preset high-risk terminology database (such as "contraindication", "lethal dose", "maximum dose"), manual review will be triggered even if the credibility is >0.5. For example, if the answer is contraindicated for pregnant women, the threshold will be skipped and the answer will be directly reviewed manually.

[0035] S4.2 For efficiency-first scenarios such as e-commerce consultations, with the practicality and conciseness of the answers as the core objectives, the thresholds of some evaluation indicators are appropriately relaxed, and the response time threshold is relaxed by 30%, from the default 1 second to 1.3 seconds. The minimum quality of the search segment is allowed to be reduced to 0.6, but the segment breakage rate of core parameters (price, inventory, after-sales) must be ≤10%.

[0036] In this embodiment, the iterative training framework in S1 also includes a dynamic update mechanism: the framework performance is evaluated periodically based on historical optimization data. When the system performance improvement rate is lower than the threshold θ after N consecutive closed-loop optimizations, the parameter update of the evaluation module and the feedback module is automatically triggered. The parameter update includes adjusting the weight of the evaluation index and the signal conversion coefficient. N is a positive integer and 1≤N≤5. The value of θ is 5%-10%.

[0037] In this embodiment, the fracture rate threshold δ and semantic similarity threshold γ in S2 are adaptively adjusted in the following way: Based on historical retrieval segment quality data and generated answer accuracy data, a sliding window algorithm is used to dynamically update δ and γ, with the window size set to 30 days and updated weekly. In high-risk scenarios, the value of δ is reduced by 10%-20%, from the default 15% to 12%-13.5%, and the value of γ is increased by 10%-20%, from the default 0.7 to 0.77-0.84. In efficiency-priority scenarios, the opposite is true: δ is increased from 15% to 16.5%-18%, and γ is reduced from 0.7 to 0.63-0.56.

[0038] This application constructs a closed-loop iterative framework of "evaluation-feedback-optimization," integrating multi-level evaluation feedback on retrieval chunk quality, generation accuracy, and user behavior. This transforms quantified results into reinforcement learning reward signals, driving the agent to autonomously adjust its strategy. Simultaneously, by combining this with an efficient path for reusing historical decision knowledge bases, it addresses the issues of information fragmentation and contradictory answers in dynamic scenarios, enabling continuous iterative improvement of system performance.

[0039] In summary, the innovation of this application lies in: (i) Construct an iterative training framework that integrates multi-level evaluation feedback and reinforcement learning decision-making, including an evaluation module, a feedback module and an optimization module, and the three modules are interconnected to form a closed loop.

[0040] (ii) Conduct a hierarchical evaluation of the retrieval process of the intelligent question-answering system, covering document structure feature analysis, text segmentation rationality evaluation, and segmentation reorganization mechanism.

[0041] (iii) Conduct hierarchical evaluation during the generation stage, including breaking down the generated answer into atomic statements, cross-referencing them with the retrieval context information, identifying and filtering fictitious, contradictory or ambiguous content, and correcting the answer for different scenarios.

[0042] (iv) Integrate the segmented quality assessment results of the retrieval process, the accuracy verification results of the answers in the generation stage, and the behavioral feedback data of end users to form a three-layer optimization signal.

[0043] (v) Based on the optimization signal-driven dynamic adjustment strategy of reinforcement learning agent, it involves reward function design, specific implementation of strategy adjustment and application of historical decision knowledge base.

[0044] (vi) Based on the risk characteristics of the application scenarios, implement differentiated threshold management for high-risk areas such as medical care and finance, and efficiency-priority scenarios such as e-commerce consulting.

[0045] This application implements differentiated threshold management based on the risk characteristics of different scenarios. It establishes strict review and manual verification mechanisms for high-risk fields such as healthcare and finance, while relaxing indicators to balance efficiency for scenarios like e-commerce consultation. Through adaptive threshold adjustment and dynamic framework updates, it balances accuracy and efficiency, significantly improving the adaptability of the intelligent question-answering system in complex scenarios.

[0046] This application has two main advantages: 1) Compared to traditional image personalization generation methods, this method does not rely on any training or fine-tuning, thus saving significant computational resources and training data requirements. This allows this application to be deployed in a plug-and-play manner to the generation process of downstream tasks without requiring additional fine-tuning based on the specific characteristics of each task.

[0047] 2) It solves the prompt-following problem in the personalized text-to-image generation process, effectively handling the control of details in the generated image from text prompts. At the same time, difference perception can effectively distinguish the modified content in the text prompts from the semantic information in the main object in the reference image, thus ensuring high fidelity of the generated image and preventing the model from uncontrollably generating images unrelated to the main object in the reference image.

[0048] Furthermore, this application introduces a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an image personalization generation method based on differential sensing fusion.

[0049] Furthermore, this application describes a computer device comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of an image personalization generation method based on differential sensing fusion.

[0050] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0051] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0052] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0053] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0054] The above is an example of the working process and results of one specific embodiment of this application, for reference only. This application is not limited to this embodiment, and other variations or equivalent substitutions are possible. All such variations or substitutions, without departing from the technical solution and inventive concept of this application, fall within the protection scope of this application.

[0055] It should be noted that the above detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0056] In the detailed description above, reference has been made to the accompanying drawings, which form part of this document. In the drawings, similar symbols typically identify similar parts unless the context otherwise indicates otherwise. The illustrated embodiments described in the detailed specification, drawings, and claims are not intended to be limiting. Other embodiments may be used and other changes may be made without departing from the spirit or scope of the subject matter presented herein.

Claims

1. A reinforcement learning optimization method for an intelligent question-answering system based on multi-level evaluation feedback, characterized in that, The method includes the following steps: S1. Construct an iterative training framework that integrates multi-level evaluation feedback and reinforcement learning decision-making; S2. Based on the constructed iterative training framework, a hierarchical evaluation of the intelligent question answering system is performed to obtain optimization signals; S3. Based on the acquired optimization signals, a dynamic adjustment strategy is used to drive the reinforcement learning agent. S4. Based on the dynamic adjustment strategy and combined with the risk characteristics of the application scenario, implement differentiated threshold management to ultimately form a closed-loop performance optimization mechanism.

2. The reinforcement learning optimization method for intelligent question answering systems based on multi-level evaluation feedback as described in claim 1, characterized in that, The construction of the iterative training framework in S1 further includes: The evaluation module is used to evaluate each part of the system. The evaluation module has a built-in multi-dimensional evaluation index library, and each index supports dynamic weight configuration. The feedback step involves converting the evaluation results into optimization signals. The signal quantizer maps the qualitative evaluation results into quantized scores, ensuring that the signals can be directly parsed by the reinforcement learning model. The optimization module adjusts the strategy based on the optimized signal drive, and the optimization steps are implemented.

3. The reinforcement learning optimization method for intelligent question answering systems based on multi-level evaluation feedback as described in claim 2, wherein the optimization module further comprises: The system includes a strategy generator and an execution engine. The strategy generator outputs specific adjustment instructions, and the execution engine interfaces with the API of the retrieval and generation components to implement the strategy.

4. The reinforcement learning optimization method for intelligent question answering systems based on multi-level evaluation feedback as described in claim 3, characterized in that: The evaluation module, feedback module, and optimization module interact through a real-time data bus, forming an iterative closed loop of evaluation data input → feedback signal output → optimization strategy execution → evaluation data update.

5. The reinforcement learning optimization method for intelligent question answering systems based on multi-level evaluation feedback as described in claim 1, characterized in that: The intelligent question-answering system further includes a retrieval module and a generation module; The retrieval module is used to obtain contextual information related to the question from the document. The generation module generates answers based on the retrieved context information, using a pre-trained language model, and the generation process supports evidence citation annotation. The information retrieval quality of the retrieval module and the answer output quality of the generation module are evaluated hierarchically.

6. The reinforcement learning optimization method for intelligent question answering systems based on multi-level evaluation feedback as described in claim 5, characterized in that: The hierarchical evaluation of the retrieval module further includes the following steps: S2.1 Analyze the structural features of the document, including the heading hierarchy tree and table anchor point positions. The heading hierarchy tree is constructed by analyzing the document outline tags. S2.

2. Based on the structural features of the parsed document, dynamically evaluate the rationality of text segmentation, calculate the breakage rate between headings and table anchor points, and calculate the breakage rate R. f The formula is: in: The number of times the title and related tables are split into different blocks; : Total number of title-table related pairs in the document; S2.3, when the fracture rate R f When the threshold δ is exceeded, a block reorganization mechanism is automatically triggered to ensure the integrity and accuracy of the context information. The block reorganization mechanism is cross-block semantic aggregation, which calculates the semantic similarity of adjacent blocks and merges blocks with a similarity ≥ 0.

85. S2.4 Decompose the generated answer into atomic statements, which is achieved through dependency parsing, breaking down complex sentences into simple statements; S2.

5. Cross-compare the obtained atomic statements with the context information obtained by the retrieval module. The cross-compare is achieved by calculating semantic similarity. After converting the atomic statements and context text into vectors, the matching degree is calculated. The formula is: Similarity = (Atomic statement vector) / (Atomic statement vector) Context vector) / (|| atomic statement vector|| × || context vector||); S2.

6. Based on the comparison results, identify and filter fictitious, contradictory, or ambiguous content, and make targeted corrections to the answers for different scenarios; S2.7 Collect behavioral feedback data from end users, including interaction duration, second question rate, and answer acceptance rate; S2.

8. Integrate the segmented quality assessment results of the retrieval module, the answer accuracy verification results of the generation module, and the collected user behavior feedback data to form a three-layer optimization signal, which is then input into the decision engine and integrated into a vector [Q]. ret Q gen Q user ], where Q ret For the retrieval quality score: (100×(1-R)) f )), Q gen To generate a quality score: (100 × average semantic similarity), Q user User feedback score: (50 × adoption rate + 30 × (1 - second question rate) + 20 × (interaction duration / average duration)).

7. The reinforcement learning optimization method for intelligent question answering systems based on multi-level evaluation feedback as described in claim 6, characterized in that: The dynamic adjustment strategy for the reinforcement learning agent in S3 further includes the following steps: S3.1 Quantify the obtained multi-level evaluation results and convert them into a reward function for reinforcement learning; The reward function is: R = α·Q ret + β·Q gen + γ·Q user , where α, β, and γ are default weights that can be dynamically adjusted according to the scenario; S3.2 Training the agent to autonomously adjust retrieval and generation strategies based on the reward function; The strategy adjustments include enhancing keyword retrieval weights and enabling multi-round searches, when Q... ret When Q < 70, increase the weight of core keywords; when Q gen When the value is less than 65, supplementary search terms are generated for questionable atomic statements, triggering multiple rounds of searches to supplement evidence; S3.

3. Accumulate efficient decision-making paths through continuous learning and build a historical decision-making knowledge base to support strategy adjustments.

8. A reinforcement learning optimization system for intelligent question answering based on multi-level evaluation feedback, characterized in that, The system includes: Build modules are used to construct an iterative training framework that integrates multi-level evaluation feedback and reinforcement learning decision-making; The evaluation and optimization module, based on the constructed iterative training framework, performs hierarchical evaluation of the intelligent question answering system to obtain optimization signals; The policy adjustment module, based on the acquired optimization signals, is used to drive the reinforcement learning agent to dynamically adjust its policy. The management module, based on dynamic adjustment strategies and combined with the risk characteristics of application scenarios, implements differentiated threshold management, ultimately forming a closed-loop performance optimization mechanism.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the reinforcement learning optimization method for the intelligent question answering system based on multi-level evaluation feedback as described in any one of claims 1 to 8.

10. A computer device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the reinforcement learning optimization method for an intelligent question-answering system based on multi-level evaluation feedback as described in any one of claims 1 to 8.

Images