A new large model autonomous reasoning system based on fast and slow thinking
By combining fast and slow thinking theory with metacognitive ability, a novel large-scale autonomous reasoning system has been developed, which solves the problems of reasoning cascade failure and difficulty in knowledge reuse in complex problems caused by language models. It achieves more efficient and accurate reasoning ability and is suitable for complex tasks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING INST OF TECH
- Filing Date
- 2025-05-30
- Publication Date
- 2026-07-21
AI Technical Summary
Existing language models face challenges such as inference cascade failure and difficulty in knowledge reuse when dealing with complex problems. In particular, they struggle to maintain consistency and accuracy in reasoning in multi-step reasoning tasks, and their knowledge utilization is inflexible when facing new tasks.
A novel large-scale autonomous reasoning system based on the fast and slow thinking theory is adopted. It combines the rapid intuition of System 1 and the logical reasoning of System 2, and introduces metacognitive ability. It achieves autonomous reasoning by rapidly integrating information through System 1 and using the logical reasoning and knowledge expansion modules of System 2. Furthermore, it optimizes the interaction process between System 1 and System 2 through reinforcement learning.
It improves the model's inference efficiency and accuracy, enhances its ability to handle complex tasks, and increases its application potential in complex and open environments.
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a novel large-scale autonomous reasoning system based on fast and slow thinking. Background Technology
[0002] With the rapid development of artificial intelligence technology, language models (such as GPT-4) have made significant progress in natural language processing, knowledge reasoning, and solving complex problems. However, despite their excellent performance in generating text and answering questions, these models still face many challenges when dealing with complex problems. Among them, the failure of reasoning cascades and the difficulty in reusing knowledge are two particularly prominent issues.
[0003] Inference cascading failure refers to a situation where, during multi-step reasoning, errors accumulate gradually because the result of each step depends on the output of the previous step, ultimately causing the overall reasoning result to deviate from the correct direction. This problem is particularly pronounced in long-chain reasoning tasks, such as mathematical proofs, complex logical reasoning, or multi-step problem solving. While existing language models perform well in single-step reasoning, they often struggle to maintain consistency and accuracy in tasks requiring multi-step reasoning.
[0004] The difficulty in reusing knowledge refers to the model's inability to effectively utilize existing knowledge and experience when dealing with new problems. Although language models acquire massive amounts of knowledge through large-scale pre-training, this knowledge is often difficult to flexibly call upon and reuse when faced with new tasks. For example, a model may perform well in solving a certain type of problem, but its performance will significantly decline when the problem changes slightly or involves cross-domain knowledge. This "rigidity" of knowledge limits the model's generalization ability and adaptability in practical applications.
[0005] To address these issues, researchers have begun to draw upon the "fast and slow thinking" theory from cognitive psychology, attempting to apply it to the construction of artificial intelligence models. This theory divides the human thought process into two systems: System 1, which rapidly integrates information, and System 2, which involves logical reasoning, knowledge expansion, and knowledge induction. In the field of artificial intelligence, this division of System 1 and System 2 provides a new approach to building more intelligent reasoning systems. By combining the rapid intuition of System 1 with the logical reasoning of System 2, the model can quickly generate preliminary solutions when dealing with complex problems, and then correct and optimize them through step-by-step reasoning. This combination not only improves the model's reasoning efficiency but also enhances its ability to handle complex tasks.
[0006] Therefore, this invention proposes a novel large-scale autonomous reasoning system based on fast and slow thinking to overcome the challenges faced by existing language models in dealing with complex problems, such as reasoning cascade failure and difficulty in knowledge reuse. Summary of the Invention
[0007] The purpose of this invention is to provide a novel autonomous reasoning system for large models based on fast and slow thinking. By drawing on the theory of fast and slow thinking, combining the rapid intuition of System 1 and the logical reasoning of System 2, and introducing metacognitive abilities, a novel autonomous reasoning system is constructed. This system can not only improve the reasoning efficiency and accuracy of the model, but also enhance its ability to handle complex tasks, providing new possibilities for the application of artificial intelligence in complex and open environments.
[0008] To achieve the above objectives, this invention provides a novel large-scale autonomous reasoning system based on fast and slow thinking, comprising a system 1 rapid information integration module, a system 2 logical reasoning and knowledge expansion module, and a knowledge induction module; wherein, the system takes metacognitive ability as its core, and combines multi-level reasoning of language models and knowledge induction modules to achieve autonomous reasoning.
[0009] Preferably, System 1 rapidly integrates the information module, utilizes a large language model, and combines background knowledge to generate questions. It is responsible for rapidly generating global information, i.e., macro-level beliefs, to provide input for the reasoning of System 2. The specific process is as follows: Step S11, Problem Description and Background Extraction: Extract core background information based on the input problem; Step S12: Generate macro-level beliefs: Make preliminary predictions on all possible solutions to the problem and provide assessments of key risk points and experience sharing.
[0010] Preferably, the logical reasoning and knowledge expansion module of System 2, based on the macro-beliefs generated by System 1, uses multiple reasoning methods to complete the step-by-step problem decomposition and solution, as well as self-reflection and error correction. The specific process is as follows: Step S21, Reasoning: Use single-step reasoning and multi-step search to perform step-by-step reasoning; Step S22, Self-reflection: After each reasoning step, System 2 self-checks for logical loopholes and omissions of key information by prompting itself.
[0011] Preferably, a System 1 strategy model based on System 1 rapidly integrates information modules to generate metacognitive knowledge and control, i.e., abstract actions; Based on the System 2 strategy model with System 2 logical reasoning and knowledge expansion modules, specific reasoning steps, i.e. specific actions, are generated.
[0012] Preferably, the knowledge summarization module is responsible for transforming the reasoning results into a reusable abstract knowledge chain, generalizing specific problems into a knowledge framework with universal significance, storing the context and reasoning path of specific problems, and supporting cross-domain knowledge transfer.
[0013] Preferably, the interaction process between System 1 and System 2 is as follows: (1) System 1 generates multiple metacognitive action nodes, and the process reward model evaluates the node scores, with the node with the highest reward score being input into System 2; (2) System 2 generates multiple reasoning action nodes, and the process reward model evaluates the node scores; (3) System 1 continues to generate multiple metacognitive action nodes until System 1 evaluates and deems the output to be the correct answer.
[0014] Preferably, the interaction between System 1 and System 2 is modeled using a Markov decision process in reinforcement learning, which is defined as a quintuple, as shown below: ; in, This is the state space, representing the metacognitive state or specific problem state during the reasoning process; This is the action space, representing the action of system 1 or system 2; Let be the state transition probability. Indicates the state Next action After transitioning to state The probability of; For the reward function, Indicates the state Next action The reward after the event; γ is a discount factor used to control the importance of future rewards, 0≤γ≤1; Therefore, the reasoning process of alternating between System 1 and System 2 is regarded as a recursive strategy problem with the goal of maximizing the cumulative reward.
[0015] Preferably, based on a reinforcement learning policy model, the recursive policy problem of System 1 and System 2 is optimized, and the specific process is as follows: Step S321, System 1 and System 2 define the strategies as follows: and Let and represent the probabilities of choosing metacognitive actions and specific reasoning actions, respectively. The optimization objective is to maximize the cumulative reward. As shown below: ; in, Rewards for each step; Discount factor; Indicates the step size; Therefore, policy optimization is performed using the policy gradient method, as shown below: ; in, Indicates the trajectory of reasoning; Represents the strategy model; θ is a parameter; Step S322: Based on the ActorCritic method, using the Actor-Critic framework, stabilize the strategy optimization process; The strategy used by Actors to generate actions Critic is used to estimate the state-value function. That is, to evaluate the current strategy; where, Critic's state-value function Indicates the state The expected cumulative reward is as follows: ; TD error is used to update Critic, as shown below: ; Therefore, the Actor's policy update formula is as follows: .
[0016] Preferably, in the hierarchical reinforcement learning of System 1 and System 2, actions are generated separately and optimized alternately, as follows: Step S331, Reward function of System 1 The contribution of metacognitive actions to reasoning tasks is shown below: ; in, Understanding quality for the problem; For planning quality; To improve the quality of abstract knowledge invocation; The estimation is performed using a process reward model, as shown below: ; in, They are respectively , and The weights; Step S332, Reward function of System 2 The quality of the specific reasoning steps is shown below: ; in, For the correctness of the reasoning; For efficiency; For clarity of steps; In mathematical reasoning tasks: ; in, They are respectively , and The weights; In step S333, during the interaction between System 1 and System 2, System 1 outputs metacognitive actions. System 2 according to Generate specific reasoning actions This ultimately forms a complete reasoning trajectory; therefore, the total reward is as follows: ; The strategy update formula is as follows: .
[0017] Preferably, a process reward model is used to evaluate node scores. As shown below: ; In the interaction between System 1 and System 2, the reward model is defined as follows: System 1 reward: based on the abstract quality score of the module combination; System 2 reward: based on the correctness score of the specific reasoning steps; Iterative update using the Bellman equation As shown below: .
[0018] Therefore, this invention adopts the above-mentioned novel large-scale autonomous reasoning system based on fast and slow thinking. By drawing on the theory of fast and slow thinking, combining the rapid intuition of System 1 and the logical reasoning of System 2, and introducing metacognitive ability, a novel autonomous reasoning system is constructed. This system can not only improve the reasoning efficiency and accuracy of the model, but also enhance its ability to handle complex tasks, providing new possibilities for the application of artificial intelligence in complex and open environments.
[0019] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0020] Figure 1 This is a schematic diagram of the fast and slow thinking theory of this invention; Figure 2 This is a flowchart illustrating the collaborative workflow of System 1 and System 2 of this invention. Figure 3 This is the interactive flowchart of System 1 and System 2 of the present invention. Detailed Implementation
[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0022] like Figure 1As shown, a novel large-scale autonomous reasoning system based on fast and slow thinking includes a System 1 module for rapid information integration, a System 2 module for logical reasoning and knowledge expansion, and a knowledge induction module. This system, with metacognitive ability at its core, combines multi-level reasoning and knowledge induction modules based on a Language Model (LLM) to address problems such as the failure of reasoning cascades and the difficulty in reusing knowledge.
[0023] Metacognition comprises two core dimensions: metacognitive knowledge and metacognitive control. Metacognitive knowledge is the understanding of one's own thinking, learning, and problem-solving processes. Metacognitive control aims to supervise, adjust, and evaluate the execution of cognitive tasks. Metacognitive abilities provide models with the capacity for self-reflection and optimization, forming a crucial foundation for building autonomous reasoning systems.
[0024] like Figure 2 As shown, the present invention discloses a novel large-scale autonomous reasoning system based on fast and slow thinking. The specific implementation process is as follows: Step S1: System 1 quickly integrates the information module, utilizes the Large Language Model (LLM), and combines background knowledge to generate questions. This is responsible for rapidly generating global information (macro-level beliefs) to provide input for System 2's reasoning. The specific process is as follows: Step S11, Problem Description and Background Extraction: Extract core background information based on the input problem. For example, describe the problem background, extract key information, and possible solutions.
[0025] Step S12: Generate macro-level beliefs: Make preliminary predictions about possible solutions to the problem and provide assessments of key risk points and experience sharing.
[0026] System 1's quick-integration information modules are intuitive and experience-driven, making them suitable for preliminary problem analysis and simple decision-making.
[0027] Step S2, System 2 Logical Reasoning and Knowledge Expansion Module, builds upon the macro-beliefs generated in System 1, employing various reasoning methods (such as CoT and MCTS) to complete step-by-step problem decomposition and solution, as well as self-reflection and error correction. The specific process is as follows: Step S21, Reasoning: Use methods such as single-query and multi-query to perform step-by-step reasoning.
[0028] Step S22, Self-reflection: After each reasoning step, System 2 self-checks for logical loopholes and omissions of key information by prompting itself.
[0029] System 2's logical reasoning and knowledge expansion module is based on deep reasoning using logic and algorithms, making it suitable for complex tasks.
[0030] Step S3, the knowledge summarization module, is responsible for transforming the reasoning results into reusable abstract knowledge chains, generalizing specific problems into knowledge frameworks with universal significance, and storing the context and reasoning path of specific problems, supporting cross-domain knowledge transfer. That is, System 1 abstracts high-level knowledge and thought chain templates based on the reasoning results of System 2 and stores them in the memory bank. The stored content includes the reasoning results and background knowledge of specific problems.
[0031] Step S31, the interaction between System 1 (fast thinking) and System 2 (slow thinking), is modeled using a Markov Decision Process (MDP) in reinforcement learning. A Markov Decision Process can be defined as a quintuple, as shown below: ; in, State space represents the metacognitive state or specific problem state during the reasoning process; The action space represents the action (module combination) of system 1 or system 2. Let be the state transition probability. Let be the state transition probability, representing the probability in state . Next action After transitioning to state The probability of; For the reward function, Indicates the state Next action The reward after the event; γ is the discount factor, used to control the importance of future rewards, 0≤γ≤1.
[0032] Therefore, the reasoning process of alternating between System 1 and System 2 is regarded as a recursive strategy problem with the goal of maximizing the cumulative reward.
[0033] Step S32: Optimize the recursive policy problem of System 1 and System 2 based on the reinforcement learning policy model.
[0034] Step S321, System 1 and System 2 define the strategies as follows: and Let and represent the probabilities of choosing metacognitive actions and specific reasoning actions, respectively. The optimization objective is to maximize the cumulative reward. As shown below: ; in, Rewards for each step; Discount factor; Indicates the step size.
[0035] Therefore, policy optimization is performed using the policy gradient method, as shown below: ; in, Indicates the trajectory of reasoning; This represents the strategy model; θ is a parameter.
[0036] Step S322: Based on the ActorCritic method, the Actor-Critic framework is used to stabilize the strategy optimization process.
[0037] The strategy used by Actors to generate actions Critic is used to estimate the state-value function. This involves evaluating the current strategy. Specifically, the Critic's state-value function... Indicates the state The expected cumulative reward is as follows: ; The Temporal Difference Error (TD) is used to update the Critic, as shown below: ; Therefore, the Actor's policy update formula is as follows: .
[0038] Step S33: In the hierarchical reinforcement learning of System 1 and System 2, actions are generated and optimized alternately.
[0039] Step S331, Reward function of System 1 The contribution of metacognitive actions to reasoning tasks is shown below: ; in, Understanding quality for the problem; For planning quality; To improve the quality of abstract knowledge invocation.
[0040] The estimation is performed using a process reward model, as shown below: ; in, They are respectively , and The weight.
[0041] Step S332, Reward function of System 2 The quality of the specific reasoning steps is shown below: ; in, For the correctness of the reasoning; For efficiency; For clarity of steps.
[0042] In mathematical reasoning tasks: ; in, They are respectively , and The weight.
[0043] In step S333, during the interaction between System 1 and System 2, System 1 outputs metacognitive actions. System 2 according to Generate specific reasoning actions This ultimately forms a complete reasoning trajectory. Therefore, the total reward is as follows: ; The strategy update formula is as follows: .
[0044] Step S34: The process reward model is used to evaluate node scores. As shown below: ; In the interaction between System 1 and System 2, the reward model is defined as follows: System 1 reward: based on the abstract quality score of the module combination; System 2 reward: based on the correctness score of the specific reasoning steps.
[0045] Iterative update using the Bellman equation As shown below: .
[0046] This invention improves the model's reasoning efficiency, knowledge integration capability, and error correction capability through the collaborative work of System 1 and System 2.
[0047] Example The System 1 strategy model, based on System 1's rapid information integration modules, generates metacognitive knowledge and control (abstract actions). Specific modules include: problem understanding, problem reconstruction, planning, recalling abstract knowledge, recalling concrete experience, recalling error cases, evaluation, reflection, and summarization. The strategy set comprises module combinations, as shown below: (1) Appropriate module combination in the early stage of reasoning: problem understanding, problem reconstruction, recalling abstract knowledge, planning, etc.
[0048] (2) Appropriate module combination in the later stage of reasoning: evaluation, reflection, summary, etc.
[0049] (3) A node represents an action, that is, a module combination.
[0050] The System 2 strategy model, based on the System 2 logical reasoning and knowledge expansion modules, generates specific reasoning steps (actions). Specific modules are determined according to the specific task; for example, in mathematical reasoning tasks, modules include: formula invocation, numerical calculation, substitution, and differentiation. The strategy set also contains combinations of modules, as shown below: (1) If the output of system 1 contains abstract knowledge, the module combination is to instantiate the abstract knowledge.
[0051] (2) If system 1 is evaluated as a simple problem, a node is more likely to contain multiple modules to quickly complete the reasoning. The module combination can be setting variables, setting equations and solving equations.
[0052] Furthermore, System 2 can be used with a finely tuned LLM on a mathematical dataset, allowing for easier control over the generation of individual inference steps.
[0053] like Figure 3 The following is an example of the interaction between System 1 and System 2. (1) System 1 generates multiple metacognitive action nodes, and the process reward model evaluates the node scores, with the node with the highest reward score being input into System 2; (2) System 2 generates multiple reasoning action nodes, and the process reward model evaluates the node scores; (3) System 1 continues to generate multiple metacognitive action nodes until System 1 evaluates and deems the output to be the correct answer.
[0054] Therefore, this invention adopts the above-mentioned novel large-scale autonomous reasoning system based on fast and slow thinking. By drawing on the theory of fast and slow thinking, combining the rapid intuition of System 1 and the logical reasoning of System 2, and introducing metacognitive ability, a novel autonomous reasoning system is constructed. This system can not only improve the reasoning efficiency and accuracy of the model, but also enhance its ability to handle complex tasks, providing new possibilities for the application of artificial intelligence in complex and open environments.
[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. A novel large-scale autonomous reasoning system based on fast and slow thinking, characterized in that: It includes System 1, which rapidly integrates information; System 2, which includes a logical reasoning and knowledge expansion module and a knowledge summarization module; among them, the system takes metacognitive ability as its core and combines multi-level reasoning of language model and knowledge summarization module to achieve autonomous reasoning; System 1 rapidly integrates information modules, utilizes a large language model, and combines background knowledge to generate questions. It is responsible for rapidly generating global information, i.e. macro beliefs, to provide input for the reasoning of System 2. The System 2 Logical Reasoning and Knowledge Expansion Module, based on the macro-beliefs generated in System 1, uses various reasoning methods to complete the step-by-step problem decomposition and solution, as well as self-reflection and error correction. Based on the System 1 strategy model that rapidly integrates information modules, metacognitive knowledge and control, i.e., abstract actions, are generated. Based on the System 2 strategy model with System 2 logical reasoning and knowledge expansion modules, specific reasoning steps, i.e. specific actions, are generated. The knowledge summarization module is responsible for transforming reasoning results into reusable abstract knowledge chains, generalizing specific problems into knowledge frameworks with universal significance, storing the context and reasoning path of specific problems, and supporting cross-domain knowledge transfer. The interaction between System 1 and System 2 is modeled using a Markov decision process in reinforcement learning, which is defined as a quintuple, as shown below: ； in, This is the state space, representing the metacognitive state or specific problem state during the reasoning process; This is the action space, representing the action of system 1 or system 2; Let be the state transition probability. Indicates the state Next action After transitioning to state The probability of; For the reward function, Indicates the state Next action The reward after the event; γ is a discount factor used to control the importance of future rewards, 0≤γ≤1; Therefore, the reasoning process of alternating between System 1 and System 2 is regarded as a recursive strategy problem with the goal of maximizing the cumulative reward.
2. The novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 1, characterized in that, System 1 rapidly integrates information modules, utilizes a large language model, and combines background knowledge to generate questions. It is responsible for quickly generating global information, i.e., macro-level beliefs, to provide input for System 2's reasoning. The specific process is as follows: Step S11, Problem Description and Background Extraction: Extract core background information based on the input problem; Step S12: Generate macro-level beliefs: Make preliminary predictions on all possible solutions to the problem and provide assessments of key risk points and experience sharing.
3. The novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 1, characterized in that, System 2's logical reasoning and knowledge expansion module, based on the macro-beliefs generated in System 1, employs various reasoning methods to complete step-by-step problem decomposition and solution, as well as self-reflection and error correction. The specific process is as follows: Step S21, Reasoning: Use single-step reasoning and multi-step search to perform step-by-step reasoning; Step S22, Self-reflection: After each reasoning step, System 2 self-checks for logical loopholes and omissions of key information by prompting itself.
4. The novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 1, characterized in that, The interaction process between System 1 and System 2 is as follows: (1) System 1 generates multiple metacognitive action nodes, and the process reward model evaluates the node scores, with the node with the highest reward score being input into System 2; (2) System 2 generates multiple reasoning action nodes, and the process reward model evaluates the node scores; (3) System 1 continues to generate multiple metacognitive action nodes until System 1 evaluates and deems the output to be the correct answer.
5. The novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 1, characterized in that, Based on the reinforcement learning policy model, the recursive policy problem of System 1 and System 2 is optimized. The specific process is as follows: Step S321, System 1 and System 2 define the strategies as follows: and Let and represent the probabilities of choosing metacognitive actions and specific reasoning actions, respectively. The optimization objective is to maximize the cumulative reward. As shown below: ； in, Rewards for each step; Discount factor; Indicates the step size; Therefore, policy optimization is performed using the policy gradient method, as shown below: ； in, Indicates the trajectory of reasoning; Represents the strategy model; θ is a parameter; Step S322: Based on the ActorCritic method, using the Actor-Critic framework, stabilize the strategy optimization process; The strategy used by Actors to generate actions Critic is used to estimate the state-value function. That is, to evaluate the current strategy; where, Critic's state-value function Indicates the state The expected cumulative reward is as follows: ； TD error is used to update Critic, as shown below: ； Therefore, the Actor's policy update formula is as follows: 。 6. The novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 5, characterized in that, In the hierarchical reinforcement learning of System 1 and System 2, actions are generated and optimized alternately, as follows: Step S331, Reward function of System 1 The contribution of metacognitive actions to reasoning tasks is shown below: ； in, Understanding quality for the problem; For planning quality; To improve the quality of abstract knowledge invocation; The estimation is performed using a process reward model, as shown below: ； in, They are respectively , and The weights; Step S332, Reward function of System 2 The quality of the specific reasoning steps is shown below: ； in, For the correctness of the reasoning; For efficiency; For clarity of steps; In mathematical reasoning tasks: ； in, They are respectively , and The weights; In step S333, during the interaction between System 1 and System 2, System 1 outputs metacognitive actions. System 2 according to Generate specific reasoning actions This ultimately forms a complete reasoning trajectory; therefore, the total reward is as follows: ； The strategy update formula is as follows: 。 7. The novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 6, characterized in that, Process reward model is used to evaluate node scores As shown below: ； In the interaction between System 1 and System 2, the reward model is defined as follows: System 1 reward: based on the abstract quality score of the module combination; System 2 reward: based on the correctness score of the specific reasoning steps; Iterative updates using the Bellman equation As shown below: 。