Multi-agent problem solving system based on reinforcement learning
By using a multi-agent problem-solving system based on reinforcement learning, the problems of low efficiency and insufficient accuracy in existing technologies are solved. It achieves efficient solutions to different types of problems and provides a variety and robust solution methods, ensuring the reliability and stability of the solutions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-17
- Publication Date
- 2026-04-14
AI Technical Summary
Existing technologies are inefficient and inaccurate in solving multi-step problems, and single models have performance limitations when they are difficult to adapt to complex and varied problem types. They cannot simultaneously take into account the diversity and robustness of solution methods.
A multi-agent problem-solving system based on reinforcement learning is adopted, including a coordinating agent, a validating agent, and multiple specialized agents. The correctness and innovativeness of the solution are balanced by a scoring function and adjustable weight parameters, and the robustness is tested by generating adversarial examples through the validating agent.
It achieves efficient solutions to different types of questions by dynamically allocating resources through multi-agent collaborative work, generating high-quality comprehensive solutions, and ensuring the reliability and stability of the solutions through robustness testing.
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent education technology, specifically to a multi-agent problem-solving system based on reinforcement learning. Background Technology
[0002] Current large language models suffer from low efficiency and insufficient accuracy in solving multi-step questions. The main reason is that existing technologies primarily rely on linear reasoning from a single model (such as CoT) or simple tree search methods (such as ToT), which struggle to simultaneously address the diversity and robustness of solutions. In particular, traditional tree search algorithms are computationally expensive, limiting search depth, and rely on the model itself to evaluate intermediate steps, lacking generalization ability. Furthermore, existing solutions typically undergo only one training iteration, resulting in limited optimization and difficulty adapting to complex and varied question types. Summary of the Invention
[0003] To address the shortcomings of existing technologies, this invention aims to provide a multi-agent problem-solving system based on reinforcement learning.
[0004] To achieve the above objectives, the present invention adopts the following technical solution: A multi-agent problem-solving system based on reinforcement learning includes a coordinating agent, a validating agent, and multiple specialized agents; The multiple specialized intelligent agents include an algebraic specialized intelligent agent, a geometry specialized intelligent agent, and a logic specialized intelligent agent. The specialized intelligent agents are used to generate candidate solutions in parallel for the problem to be solved, and then evaluate each candidate solution using a scoring function Q(a)=α×accuracy+β×innovation, where α and β are adjustable weight parameters; the algebraic specialized intelligent agent, the geometry specialized intelligent agent, and the logic specialized intelligent agent each have independent neural network structures and training datasets; The coordinating agent is used to assign the questions to be solved to the corresponding specialized agents using the matching degree evaluation formula C=argmax(P1,P2,P3), where C represents the final selected specialized agent, argmax represents the maximum value selection function, and P1, P2, and P3 represent the matching degree of the algebra specialized agent, the geometry specialized agent, and the logic specialized agent to the questions to be solved, respectively.
[0005] Furthermore, the coordinating agent first uses a pre-trained neural network model to analyze the probability that the question type of the problem to be solved belongs to geometry proof, algebra, or logic, as the matching degree between the problem to be solved and the geometry-specialized agent, algebra-specialized agent, and logic-specialized agent.
[0006] Furthermore, in the scoring function, the weight parameter α ranges from 0.6 to 0.8, the weight parameter β ranges from 0.2 to 0.4, and α + β = 1.
[0007] Furthermore, the professional intelligent agent calculates the scores of each candidate solution through a scoring function and selects the best solution from them. When multiple candidate solutions have the same score or all scores are higher than a preset high threshold, the multiple candidate solutions with the same score or all scores are higher than the preset high threshold are merged to generate a weighted comprehensive solution.
[0008] Furthermore, the specific process of the solution fusion is as follows: the fusion weight Wi=softmax(si / τ) of each candidate solution is calculated using an attention mechanism, where si is the solution score and τ is the temperature parameter; then, the fusion weights of each candidate solution are used to fuse the candidate solutions to generate a weighted comprehensive solution scheme; finally, a contribution analysis report of each candidate solution is output.
[0009] Furthermore, the aforementioned multi-agent problem-solving system based on reinforcement learning also includes a verification agent. This verification agent is used to generate adversarial examples to test the robustness of the final solution generated by the professional agent, and to calculate the reliability index R of the solution, where R = 1 - number of failure cases / number of test rounds. The specific process is as follows: (1) Automated adversarial testing: generating adversarial test cases such as boundary value test cases and abnormal input test cases; (2) Adaptive stress testing: (2.1) Automatically determine the number of test rounds based on the complexity of the problem; (2.2) Use the boundary value test cases generated in step (1) to perform boundary value testing, and use the abnormal input test cases generated in step (1) to perform abnormal input testing; (2.3) Robustness evaluation index: the reliability index R of the solution calculated by the verification agent, the formula is R = 1 - number of failure cases / number of test rounds; When the reliability index R reaches or exceeds the preset reliability index threshold, the solution is deemed to have passed the test. When the calculated reliability index R is lower than the reliability index threshold, the solution test is deemed unsuccessful. (3) The solutions that fail the test and their reliability index are fed back to the corresponding professional intelligent agents in real time. The professional intelligent agents then fine-tune and continuously optimize the model accordingly, forming a closed-loop improvement process.
[0010] Furthermore, the reliability index threshold is dynamically adjusted using the optimization algorithm Rthreshold = 0.9 × current threshold + 0.1 × latest pass rate.
[0011] The beneficial effects of this invention are as follows: 1. Intelligent allocation: This invention configures multiple specialized agents for different question types and dynamically allocates questions by coordinating the agents to calculate the matching degree formula (C=argmax(P1,P2,P3)). Different specialized agents can be used to solve different question types, thereby avoiding the performance limitations of a single model when handling multiple types of questions.
[0012] 2. Solution Optimization: In this invention, the professional intelligent agent balances accuracy and innovation through a scoring function (Q(a)=α×accuracy+β×innovation) and adapts to different question types through adjustable weight parameters. In addition, when faced with multiple high-quality solutions, this invention further integrates the advantages of each solution through scheme fusion to obtain better overall results.
[0013] 3. Robust verification: In this invention, the robustness of the solution is ensured by verifying the agent's generation of adversarial examples and calculating the reliability index (R=1-number of failure cases / total number of tests). Detailed Implementation
[0014] The present invention will be further described below. It should be noted that this embodiment is based on the present technical solution and provides detailed implementation methods and specific operation processes, but the protection scope of the present invention is not limited to this embodiment.
[0015] This embodiment provides a multi-agent problem-solving system based on reinforcement learning, including a coordinating agent, a validating agent, and multiple specialized agents; The multiple specialized agents include an algebraic agent, a geometry agent, and a logic agent. These agents are used to generate candidate solutions in parallel for the problem to be solved. Then, a scoring function Q(a) = α × accuracy + β × novelty is used to evaluate each candidate solution and select the optimal solution, where α and β are adjustable weight parameters. By setting different specialized agents, it can be ensured that each type of problem has a corresponding specialized agent to solve it, avoiding the performance limitations of a single model when handling multiple types of problems.
[0016] The coordinating agent assigns the problem to the corresponding specialized agent using the matching degree evaluation formula C=argmax(P1,P2,P3), where C represents the finally selected specialized agent, argmax represents the maximum value selection function, and P1, P2, and P3 represent the matching degree of the algebra specialized agent, geometry specialized agent, and logic specialized agent for the problem to be solved, respectively. For example, when P1=0.7, P2=0.9, and P3=0.5, the coordinating agent preferentially selects the geometry specialized agent to solve the problem.
[0017] In this embodiment, the coordinating agent first uses a pre-trained neural network model to analyze the probability that the question type belongs to geometry proof, algebra, or logic, as the matching degree between the question and the geometry-specialized agent, algebra-specialized agent, or logic-specialized agent. Logic questions refer to questions involving reasoning and judgment.
[0018] In this embodiment, the algebra-specialized intelligent agent, the geometry-specialized intelligent agent, and the logic-specialized intelligent agent each have independent neural network structures and training datasets.
[0019] In this embodiment, the weight parameter α in the scoring function ranges from 0.6 to 0.8, the weight parameter β ranges from 0.2 to 0.4, and α + β = 1.
[0020] The setting of weight parameters α and β in the scoring function can ensure a reasonable balance between correctness and innovativeness when evaluating solutions, and avoid a single indicator dominating the evaluation results.
[0021] For example, when a professional agent generates two solutions, a traditional solution and an innovative solution, where the traditional solution has an accuracy of 0.95 and an innovation of 0.2, its scoring function is calculated as Q = 0.7 × 0.95 + 0.3 × 0.2 = 0.725, while the innovative solution has an accuracy of 0.8 and an innovation of 0.9, its scoring function is calculated as Q = 0.7 × 0.8 + 0.3 × 0.9 = 0.83. Therefore, the professional agent will choose the innovative solution with the higher Q value, reflecting the reasonable weight of innovation under the constraint of α + β = 1.
[0022] In this embodiment, the specialized intelligent agent calculates the scores of each candidate solution using a scoring function and selects the best solution. When multiple candidate solutions have the same score or all scores are higher than a preset high threshold, the multiple candidate solutions with the same score or all scores higher than the preset high threshold are fused to generate a weighted comprehensive solution.
[0023] Furthermore, in this embodiment, the specific process of scheme fusion is as follows: the fusion weight Wi=softmax(si / τ) of each candidate solution is calculated using an attention mechanism, where si is the solution score and τ is the temperature parameter; then, the fusion weights of each candidate solution are used to fuse each candidate solution to generate a weighted comprehensive solution scheme; finally, a contribution analysis report of each candidate solution is output.
[0024] Through the above process, high-quality solutions can be selected using the scoring function. When multiple high-quality solutions exist, the advantages of each solution can be utilized through solution fusion to achieve a better overall effect.
[0025] In this embodiment, the multi-agent problem-solving system further includes a verification agent. The verification agent generates adversarial examples to test the robustness of the final solution generated by the professional agent and calculates the reliability index R of the solution, where R = 1 - number of failure cases / number of test rounds. The specific process is as follows: (1) Automated adversarial testing: generating adversarial test cases such as boundary value test cases and abnormal input test cases; Take a geometry proof problem as an example: 1. Boundary value test cases: When testing the triangle angle sum theorem, input the cases where the sum of the angles is 179.9° or 180.1°; When testing the Pythagorean theorem, input the case where the length of the right-angled side approaches 0.
[0026] 2. Exceptional input test cases: Input the preconditions for non-Euclidean geometry in the geometric proof; Test the real field when complex solutions are input into algebraic equations.
[0027] The purpose of adversarial test cases is to comprehensively verify the robustness of the solutions, ensuring that the solutions generated by the professional agent not only work correctly under normal conditions but also maintain stability and reliability under various boundary and abnormal situations. This systematic testing method verifies that the agent can effectively identify potential defects in the solutions and promotes continuous optimization of the professional agent through a closed-loop feedback mechanism.
[0028] (2) Adaptive stress testing: (2.1) The number of test rounds is automatically determined based on the complexity of the questions. For example, basic questions are tested for 50 rounds, while comprehensive questions are tested for 100 rounds. Basic questions refer to single geometric proof questions, algebraic questions, and logic questions, while comprehensive questions refer to questions involving multiple features of geometric proofs, algebra, and logic.
[0029] (2.2) Use the boundary value test cases generated in step (1) to perform boundary value testing: verify the critical cases of the input parameters, such as special cases where the coefficients in the algebraic equation approach 0 or the angle value in the geometric problem approaches 180°. Use the abnormal input test cases generated in step (1) to perform abnormal input testing: simulate various abnormal input scenarios, including abnormal conditions such as data type errors and logical structure contradictions.
[0030] (2.3) Robustness evaluation index: the reliability index R of the agent's computation solution, which is calculated by the formula R = 1 - number of failure cases / number of test rounds.
[0031] The solution is considered to have passed the test when the reliability index R reaches or exceeds the preset reliability index threshold. Specifically, the reliability index threshold can be dynamically adjusted using the optimization algorithm Rthreshold = 0.9 × current threshold + 0.1 × latest pass rate.
[0032] When the calculated reliability index R is lower than the reliability index threshold, the solution test is deemed unsuccessful.
[0033] (3) The solutions that fail the test and their reliability index are fed back to the corresponding professional intelligent agents in real time. The professional intelligent agents then fine-tune and continuously optimize the model accordingly, forming a closed-loop improvement process.
[0034] In the above process, the verification agent ensures the reliability and robustness of the solution through automated testing and dynamic optimization mechanisms, without the need for human intervention.
[0035] For those skilled in the art, various corresponding changes and modifications can be made based on the above technical solutions and concepts, and all such changes and modifications should be included within the protection scope of the claims of this invention.
Claims
1. A multi-agent problem-solving system based on reinforcement learning, characterized in that, This includes a coordinating agent, a validating agent, and multiple specialized agents; The multiple specialized intelligent agents include an algebraic specialized intelligent agent, a geometry specialized intelligent agent, and a logic specialized intelligent agent. The specialized intelligent agents are used to generate candidate solutions in parallel for the problem to be solved, and then evaluate each candidate solution using a scoring function Q(a)=α×accuracy+β×innovation, where α and β are adjustable weight parameters; the algebraic specialized intelligent agent, the geometry specialized intelligent agent, and the logic specialized intelligent agent each have independent neural network structures and training datasets; The coordinating agent is used to assign the questions to be solved to the corresponding specialized agents using the matching degree evaluation formula C=argmax(P1,P2,P3), where C represents the final selected specialized agent, argmax represents the maximum value selection function, and P1, P2, and P3 represent the matching degree of the algebra specialized agent, the geometry specialized agent, and the logic specialized agent to the questions to be solved, respectively.
2. The multi-agent problem-solving system based on reinforcement learning according to claim 1, characterized in that, The coordinating agent first uses a pre-trained neural network model to analyze the probability that the question type belongs to geometry proof, algebra, or logic, and uses this probability as the matching degree between the question and the geometry-specific agent, algebra-specific agent, or logic-specific agent.
3. The multi-agent problem-solving system based on reinforcement learning according to claim 1, characterized in that, In the scoring function, the weight parameter α ranges from 0.6 to 0.8, the weight parameter β ranges from 0.2 to 0.4, and α + β = 1.
4. The multi-agent problem-solving system based on reinforcement learning according to claim 1, characterized in that, The professional intelligent agent calculates the scores of each candidate solution through a scoring function and selects the best solution from them. When multiple candidate solutions have the same score or all scores are higher than a preset high threshold, the multiple candidate solutions with the same score or all scores are higher than the preset high threshold are merged to generate a weighted comprehensive solution.
5. The multi-agent problem-solving system based on reinforcement learning according to claim 4, characterized in that, The specific process of the solution fusion is as follows: the fusion weight Wi=softmax(si / τ) of each candidate solution is calculated using an attention mechanism, where si is the solution score and τ is the temperature parameter; then, the fusion weights of each candidate solution are used to fuse the candidate solutions to generate a weighted comprehensive solution scheme; finally, a contribution analysis report of each candidate solution is output.
6. The multi-agent problem-solving system based on reinforcement learning according to claim 1, characterized in that, It also includes a verification agent, which is used to generate adversarial examples to test the robustness of the final solution generated by the professional agent, and calculates the reliability index R of the solution, R = 1 - number of failure cases / number of test rounds; the specific process is as follows: (1) Automated adversarial testing: generating adversarial test cases such as boundary value test cases and abnormal input test cases; (2) Adaptive stress testing: (2.1) Automatically determine the number of test rounds based on the complexity of the problem; (2.2) Use the boundary value test cases generated in step (1) to perform boundary value testing, and use the abnormal input test cases generated in step (1) to perform abnormal input testing; (2.3) Robustness evaluation index: the reliability index R of the solution calculated by the verification agent, the formula is R = 1 - number of failure cases / number of test rounds; When the reliability index R reaches or exceeds the preset reliability index threshold, the solution is deemed to have passed the test. When the calculated reliability index R is lower than the reliability index threshold, the solution test is deemed unsuccessful. (3) The solutions that fail the test and their reliability index are fed back to the corresponding professional intelligent agents in real time. The professional intelligent agents then fine-tune and continuously optimize the model accordingly, forming a closed-loop improvement process.
7. The multi-agent problem-solving system based on reinforcement learning according to claim 6, characterized in that, The reliability index threshold is dynamically adjusted using the optimization algorithm Rthreshold = 0.9 × current threshold + 0.1 × latest pass rate.