Novel large model autonomous reasoning system based on fast and slow thinking

Abstract

The invention belongs to the technical field of artificial intelligence, and discloses a novel large model autonomous reasoning system based on fast and slow thinking, which comprises a system 1 fast integration information module, a system 2 logical reasoning and knowledge extension module and a knowledge induction module, wherein the system takes meta-cognitive competence as a core, and realizes autonomous reasoning in combination with a multi-level reasoning and knowledge induction module of a language model. According to the novel large model autonomous reasoning system based on fast and slow thinking, the fast and slow thinking theory is used for reference, the fast intuition of the system 1 and the logical reasoning of the system 2 are combined, the meta-cognitive ability is introduced, the novel autonomous reasoning system is constructed, the reasoning efficiency and accuracy of the model can be improved, and the reasoning efficiency and accuracy of the model can be improved. And the capability of processing complex tasks can be enhanced, and a new possibility is provided for application of artificial intelligence in a complex open environment.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a novel large-model autonomous reasoning system based on fast and slow thinking. Background Art

[0002] With the rapid development of artificial intelligence (AI), language models (such as GPT-4) have made significant progress in natural language processing, knowledge reasoning, and complex problem solving. However, despite their impressive performance in generating text and answering questions, these models still face numerous challenges when handling complex problems. Among these, cascading failures in reasoning and the difficulty in reusing knowledge are two particularly prominent issues.

[0003] Cascading reasoning failure occurs when a model performs multi-step reasoning. Because the results of each step depend on the output of the previous step, errors accumulate during the reasoning process, ultimately causing the overall reasoning result to deviate from the correct direction. This problem is particularly evident in long-chain reasoning tasks, such as mathematical proofs, complex logical reasoning, or multi-step problem solving. While existing language models perform well for single-step reasoning, they often struggle to maintain consistency and accuracy in tasks requiring multi-step reasoning.

[0004] Difficulty in knowledge reuse refers to the model's inability to effectively utilize existing knowledge and experience when solving new problems. Although language models acquire vast amounts of knowledge through large-scale pre-training, this knowledge is often difficult to flexibly call upon and reuse when faced with new tasks. For example, a model may perform well for a certain type of problem, but its performance will significantly decline when the problem changes slightly or involves cross-domain knowledge. This "rigidity" of knowledge limits the model's generalization and adaptability in practical applications.

[0005] To address these issues, researchers have begun drawing on the "fast and slow thinking" theory from cognitive psychology and attempting to apply it to the construction of artificial intelligence models. This theory divides the human thinking process into two systems: System 1, which rapidly integrates information, and System 2, which logically reasoned, expanded, and summarized knowledge. In the field of artificial intelligence, the division of System 1 and System 2 provides new insights into building more intelligent reasoning systems. By combining the rapid intuition of System 1 with the logical reasoning of System 2, the model can quickly generate preliminary solutions to complex problems while also performing error correction and optimization through step-by-step reasoning. This combination not only improves the model's reasoning efficiency but also enhances its ability to handle complex tasks.

[0006] Therefore, the present invention proposes a new large-model autonomous reasoning system based on fast and slow thinking to overcome the challenges faced by existing language models when dealing with complex problems, such as reasoning cascade failure and difficulty in knowledge reuse. Summary of the Invention

[0007] The purpose of this invention is to provide a new large-model autonomous reasoning system based on fast and slow thinking. By drawing on the theory of fast and slow thinking, combining the fast intuition of System 1 and the logical reasoning of System 2, and introducing metacognitive ability, a new type of autonomous reasoning system is constructed. This system can not only improve the reasoning efficiency and accuracy of the model, but also enhance its ability to handle complex tasks, providing new possibilities for the application of artificial intelligence in complex open environments.

[0008] To achieve the above objectives, the present invention provides a new large-model autonomous reasoning system based on fast and slow thinking, including a system 1 rapid information integration module, a system 2 logical reasoning and knowledge expansion module, and a knowledge induction module; wherein, the system takes metacognitive ability as the core, combines the multi-level reasoning and knowledge induction modules of the language model to realize autonomous reasoning.

[0009] Preferably, System 1 quickly integrates information modules, uses a large language model, and generates questions based on background knowledge. It is responsible for quickly generating global information, namely macro beliefs, to provide input for the reasoning of System 2. The specific process is as follows:

[0010] Step S11, problem description and background extraction: extracting core background information based on the input problem;

[0011] Step S12: Generate macro-beliefs: Make preliminary predictions on all solutions to the problem, and provide assessments and experience sharing on key risk points.

[0012] Preferably, the logical reasoning and knowledge expansion module of System 2 uses multiple reasoning methods based on the macro beliefs generated by System 1 to complete step-by-step problem decomposition and solution, self-reflection, and error correction. The specific process is as follows:

[0013] Step S21, reasoning: using single-step reasoning and multi-step search to perform step-by-step reasoning;

[0014] Step S22, self-reflection: After each reasoning, System 2 prompts itself to check whether there are logical loopholes and whether key information is omitted.

[0015] Preferably, the System 1 strategy model based on the System 1 rapid integration of information modules generates metacognitive knowledge and control, i.e., abstract action;

[0016] The System 2 strategy model based on the System 2 logical reasoning and knowledge expansion module generates specific reasoning steps, that is, specific actions.

[0017] Preferably, the knowledge induction module is responsible for converting the reasoning results into reusable abstract knowledge chains, generalizing specific problems into knowledge frameworks with general meanings, and storing the context and reasoning paths of specific problems to support cross-domain knowledge transfer.

[0018] Preferably, the interaction process between system 1 and system 2 is as follows:

[0019] (1) System 1 generates multiple metacognitive action nodes, and the process reward model evaluates the node scores, among which the node with the highest reward score is input into System 2;

[0020] (2) System 2 generates multiple reasoning action nodes, and the process reward model evaluates the node scores;

[0021] (3) System 1 continues to generate multiple metacognitive action nodes until System 1 evaluates and outputs the correct answer.

[0022] Preferably, the interaction between system 1 and system 2 is modeled by a Markov decision process in reinforcement learning. The Markov decision process is defined as a five-tuple as follows:

[0023] M=(S,A,P,R,γ);

[0024] Where S is the state space, which represents the metacognitive state or specific problem state in the reasoning process; A is the action space, which represents the action of System 1 or System 2; P is the state transition probability, P(s'|s,a) represents the probability of transitioning to state s' after executing action a in state s; R is the reward function, R(s,a) represents the reward after executing action a in state s; γ is the discount factor used to control the importance of future rewards, 0≤γ≤1;

[0025] Therefore, the reasoning process performed alternately by System 1 and System 2 can be regarded as a recursive policy problem, where the goal is to maximize the cumulative reward.

[0026] Preferably, based on the reinforcement learning strategy model, the recursive strategy problem of system 1 and system 2 is optimized. The specific process is as follows:

[0027] Step S321, system 1 and system 2 define strategies as π1(a|s) and π2(a|s), which represent the selection probabilities of metacognitive actions and specific reasoning actions respectively. The optimization goal is to maximize the cumulative reward G t , as shown below:

[0028] G t =∑ ∞ γ k R(s t+k ,a t+k );

[0029] Among them, R(s t+k ,a t+k ) is the reward for each step; γ is the discount factor; k represents the step length;

[0030] Therefore, policy optimization is performed through the policy gradient method as follows:

[0031]

[0032] Among them, τ represents the reasoning trajectory; π represents the policy model; θ is the parameter;

[0033] Step S322: Based on the ActorCritic method, the Actor-Critic framework is adopted to stabilize the strategy optimization process;

[0034] Actor's strategy for generating actions π θ (a|s); Critic is used to estimate the state value function V(s), that is, to evaluate the current strategy; where the Critic's state value function V(s) represents the expected cumulative reward in state s, as shown below:

[0035]

[0036] The TD error is used to update the Critic as follows:

[0037] δ t =R(s t ,a t )+γV(s t+1 )-V(s t );

[0038] Therefore, the Actor's strategy update formula is as follows:

[0039]

[0040] Preferably, in the hierarchical reinforcement learning of system 1 and system 2, each generates actions and optimizes alternately. The specific process is as follows:

[0041] Step S331, the reward function R1 of system 1 reflects the contribution of metacognitive actions to the reasoning task, as shown below:

[0042] R1=f(l,p,q);

[0043] Among them, l is the problem understanding quality; p is the planning quality; q is the abstract knowledge call quality;

[0044] The estimation is done through the process reward model as follows:

[0045] R1=α1·Q l +α2·Qp +α3·Q q ;

[0046] Among them, α1, α2, and α3 are Q l , Q p and Q q The weight of

[0047] In step S332, the reward function R2 of system 2 reflects the quality of the specific reasoning step, as shown below:

[0048] R2=f(t,x,b);

[0049] Among them, t is the correctness of reasoning; x is efficiency; b is the clarity of steps;

[0050] In mathematical reasoning tasks:

[0051] R2=β1·Q t +β2·Q x +β3·Q b ;

[0052] Among them, β1, β2, and β3 are Q t , Q x and Q b The weight of

[0053] In step S333, during the interaction between system 1 and system 2, system 1 outputs a metacognitive action a1, and system 2 generates a specific reasoning action a2 based on a1, ultimately forming a complete reasoning trajectory; therefore, the total reward is as follows:

[0054] R total =R1+R2;

[0055] The strategy update formula is as follows:

[0056]

[0057] Preferably, a process reward model is used to evaluate the node score Q(s,a) as follows:

[0058]

[0059] In the interaction between System 1 and System 2, the reward model is defined as follows:

[0060] System 1 rewards: scores based on the abstract quality of module combinations; System 2 rewards: scores based on the correctness of specific reasoning steps;

[0061] Q is iteratively updated using the Bellman equation as follows:

[0062]

[0063] Therefore, the present invention adopts the above-mentioned new large-model autonomous reasoning system based on fast and slow thinking. By drawing on the theory of fast and slow thinking, combining the fast intuition of system 1 and the logical reasoning of system 2, and introducing metacognitive ability, a new autonomous reasoning system is constructed. This system can not only improve the reasoning efficiency and accuracy of the model, but also enhance its ability to handle complex tasks, providing new possibilities for the application of artificial intelligence in complex open environments.

[0064] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a schematic diagram of the fast and slow thinking theory of the present invention;

[0066] Figure 2 This is a diagram of the collaborative work flow of system 1 and system 2 of the present invention;

[0067] Figure 3 It is the interactive flow chart of system 1 and system 2 of the present invention. DETAILED DESCRIPTION

[0068] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0069] like Figure 1 As shown in the figure, a new large-scale autonomous reasoning system based on fast and slow thinking consists of a System 1 rapid information integration module and a System 2 logical reasoning and knowledge expansion module and knowledge summarization module. This system, with metacognitive capabilities at its core, combines the multi-level reasoning and knowledge summarization modules of the language model (LLM) to address issues such as cascading reasoning failures and difficulty in knowledge reuse.

[0070] Metacognition encompasses two core dimensions: metacognitive knowledge and metacognitive control. Metacognitive knowledge is the understanding of one's own thinking, learning, and problem-solving processes. Metacognitive control aims to monitor, adjust, and evaluate the performance of cognitive tasks. Metacognitive ability provides models with the capacity for self-reflection and optimization, forming an important foundation for building autonomous reasoning systems.

[0071] like Figure 2 As shown, the present invention is a new large-model autonomous reasoning system based on fast and slow thinking. The specific implementation process is as follows:

[0072] In step S1, System 1 quickly integrates information modules, uses the Large Language Model (LLM), and combines background knowledge to generate questions. It is responsible for quickly generating global information (macro-beliefs) to provide input for the reasoning of System 2. The specific process is as follows:

[0073] Step S11, Problem Description and Background Extraction: Extract core background information based on the input problem, for example, describing the problem background, extracting key information, and possible solutions.

[0074] Step S12: Generate macro-beliefs: Make preliminary predictions on possible solutions to the problem, and provide assessments and experience sharing on key risk points.

[0075] System 1 quickly integrates information modules that are intuitive and experience-driven, suitable for preliminary problem analysis and simple decision-making.

[0076] Step S2, the logical reasoning and knowledge expansion module of System 2, based on the macro-beliefs generated by System 1, uses multiple reasoning methods (such as CoT and MCTS) to complete step-by-step problem decomposition and solution, as well as self-reflection and error correction. The specific process is as follows:

[0077] Step S21, reasoning: single-query and multi-query methods are used to perform step-by-step reasoning.

[0078] Step S22, self-reflection: After each reasoning, System 2 prompts itself to check whether there are logical loopholes and whether key information is omitted.

[0079] System 2 logical reasoning and knowledge expansion module is based on deep reasoning of logic and algorithms and is suitable for complex tasks.

[0080] Step S3, the knowledge summarization module, is responsible for converting reasoning results into reusable abstract knowledge chains, generalizing specific problems into a universal knowledge framework, and storing the context and reasoning paths of specific problems, supporting cross-domain knowledge transfer. Specifically, based on the reasoning results of System 2, System 1 abstracts high-level knowledge and thought chain templates and stores them in a memory bank. This storage includes the reasoning results and background knowledge for the specific problem.

[0081] In step S31, the interaction between System 1 (fast thinking) and System 2 (slow thinking) is modeled using a Markov decision process (MDP) in reinforcement learning. The MDP can be defined as a five-tuple as follows:

[0082] M=(S,A,P,R,γ);

[0083] Among them, S is the state space, which represents the metacognitive state or specific problem state in the reasoning process; A is the action space, which represents the action (module combination) of System 1 or System 2; P is the state transition probability, P(s'|s,a), which represents the probability of transitioning to state s' after performing action a in state s; R is the reward function, R(s,a) represents the reward after performing action a in state s; γ is the discount factor, which is used to control the importance of future rewards, 0≤γ≤1.

[0084] Therefore, the reasoning process performed alternately by System 1 and System 2 can be regarded as a recursive policy problem, where the goal is to maximize the cumulative reward.

[0085] Step S32: Based on the reinforcement learning strategy model, the recursive strategy problems of system 1 and system 2 are optimized.

[0086] Step S321, system 1 and system 2 define strategies as π1(a|s) and π2(a|s), which represent the selection probabilities of metacognitive actions and specific reasoning actions respectively. The optimization goal is to maximize the cumulative reward G t , as shown below:

[0087] G t =∑ ∞ γ k R(s t+k ,a t+k );

[0088] Among them, R(s t+k ,a t+k ) is the reward for each step; γ is the discount factor; k represents the step length.

[0089] Therefore, policy optimization is performed through the policy gradient method as follows:

[0090]

[0091] Among them, τ represents the inference trajectory; π represents the policy model; and θ is a parameter.

[0092] Step S322: Based on the ActorCritic method, the Actor-Critic framework is adopted to stabilize the strategy optimization process.

[0093] Actor's strategy for generating actions π θ (a|s); Critic is used to estimate the state value function V(s), that is, to evaluate the current policy. Among them, the state value function V(s) of Critic represents the expected cumulative reward in state s, as shown below:

[0094]

[0095] The TD error (Temporal Difference Error) is used to update the Critic as follows:

[0096] δ t =R(s t ,a t )+γV(s t+1 )-V(s t );

[0097] Therefore, the Actor's strategy update formula is as follows:

[0098]

[0099] Step S33: In the hierarchical reinforcement learning of system 1 and system 2, each generates actions and optimizes them alternately.

[0100] Step S331, the reward function R1 of system 1 reflects the contribution of metacognitive actions to the reasoning task, as shown below:

[0101] R1=f(l,p,q);

[0102] Where l is the quality of problem understanding, p is the quality of planning, and q is the quality of abstract knowledge retrieval. It is estimated using the process reward model as follows:

[0103] R1=α1·Q l +α2·Q p +α3·Q q ;

[0104] Among them, α1, α2, and α3 are Q l , Q p and Q q The weight of .

[0105] In step S332, the reward function R2 of system 2 reflects the quality of the specific reasoning step, as shown below:

[0106] R2=f(t,x,b);

[0107] Among them, t is the correctness of reasoning; x is efficiency; and b is the clarity of steps.

[0108] In mathematical reasoning tasks:

[0109] R2=β1·Q t +β2·Q x +β3·Q b ;

[0110] Among them, β1, β2, and β3 are Q t , Q x and Q b The weight of .

[0111] In step S333, during the interaction between system 1 and system 2, system 1 outputs metacognitive action a1, and system 2 generates specific reasoning action a2 based on a1, ultimately forming a complete reasoning trajectory. Therefore, the total reward is as follows:

[0112] R total =R1+R2;

[0113] The strategy update formula is as follows:

[0114]

[0115] Step S34: The process reward model is used to evaluate the node score Q(s,a), as shown below:

[0116]

[0117] In the interaction between System 1 and System 2, the reward model is defined as follows:

[0118] System 1 rewards: scoring based on the abstract quality of module combinations; System 2 rewards: scoring based on the correctness of specific reasoning steps.

[0119] Q is iteratively updated using the Bellman equation as follows:

[0120]

[0121] The present invention improves the reasoning efficiency, knowledge integration capability and error correction capability of the model through the collaborative work of system 1 and system 2.

[0122] Example

[0123] The System 1 strategy model, based on the System 1 rapid information integration module, generates metacognitive knowledge and control (abstract action). Specific modules include: problem understanding, problem reconstruction, planning, recalling abstract knowledge, recalling specific experiences, recalling error cases, evaluation, reflection, and summary. The strategy set includes a combination of modules as follows:

[0124] (1) Appropriate module combination in the early stage of reasoning: problem understanding, problem reconstruction, recalling abstract knowledge, planning, etc.

[0125] (2) Appropriate module combination in the later stage of reasoning: evaluation, reflection, summary, etc.

[0126] (3) A node represents an action, that is, a module combination.

[0127] The System 2 strategy model, based on the System 2 logical reasoning and knowledge expansion modules, generates specific reasoning steps (specific actions). Specific modules are determined based on specific tasks. For example, in mathematical reasoning tasks, modules include: formula calling, numerical operations, variable substitution, derivative calculation, etc. The strategy set also includes a combination of modules, as shown below:

[0128] (1) If the output of system 1 contains abstract knowledge, the modules are combined to instantiate the abstract knowledge.

[0129] (2) If the system 1 is evaluated as a simple problem, a node is more likely to contain multiple modules to quickly complete the reasoning. The module combination can be setting variables, writing equations, and solving equations.

[0130] In addition, System 2 can use LLMs fine-tuned on mathematical datasets to more conveniently control the generation of individual reasoning steps.

[0131] like Figure 3 The following is an example of the interaction between System 1 and System 2:

[0132] (1) System 1 generates multiple metacognitive action nodes, and the process reward model evaluates the node scores, among which the node with the highest reward score is input into System 2;

[0133] (2) System 2 generates multiple reasoning action nodes, and the process reward model evaluates the node scores;

[0134] (3) System 1 continues to generate multiple metacognitive action nodes until System 1 evaluates and outputs the correct answer.

[0135] Therefore, the present invention adopts the above-mentioned new large-model autonomous reasoning system based on fast and slow thinking. By drawing on the theory of fast and slow thinking, combining the fast intuition of system 1 and the logical reasoning of system 2, and introducing metacognitive ability, a new autonomous reasoning system is constructed. This system can not only improve the reasoning efficiency and accuracy of the model, but also enhance its ability to handle complex tasks, providing new possibilities for the application of artificial intelligence in complex open environments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A new large-scale autonomous reasoning system based on fast and slow thinking, characterized by: It includes the system 1 rapid information integration module, the system 2 logical reasoning and knowledge expansion module and the knowledge induction module; among them, the system takes metacognitive ability as the core, combines the multi-level reasoning and knowledge induction modules of the language model to achieve autonomous reasoning.

2. A novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 1, characterized in that: System 1 quickly integrates information modules, uses a large language model, and combines background knowledge to generate questions. It is responsible for quickly generating global information, namely macro beliefs, to provide input for the reasoning of System 2. The specific process is as follows: Step S11, problem description and background extraction: extracting core background information based on the input problem; Step S12: Generate macro-beliefs: Make preliminary predictions on all solutions to the problem, and provide assessments and experience sharing on key risk points.

3. A novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 1, characterized in that: System 2 logical reasoning and knowledge expansion module, based on the macro-beliefs generated by System 1, uses multiple reasoning methods to complete step-by-step problem decomposition and solution, as well as self-reflection and error correction. The specific process is as follows: Step S21, reasoning: using single-step reasoning and multi-step search to perform step-by-step reasoning; Step S22, self-reflection: After each reasoning, System 2 prompts itself to check whether there are logical loopholes and whether key information is omitted.

4. The novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 1 is characterized by: The System 1 strategy model, based on the System 1 rapid information integration module, generates metacognitive knowledge and control, i.e., abstract action; The System 2 strategy model based on the System 2 logical reasoning and knowledge expansion module generates specific reasoning steps, that is, specific actions.

5. The novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 1 is characterized by: The knowledge induction module is responsible for converting reasoning results into reusable abstract knowledge chains, generalizing specific problems into knowledge frameworks with general meanings, and storing the context and reasoning paths of specific problems to support cross-domain knowledge transfer.

6. A novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 1, characterized in that: The interaction between System 1 and System 2 is as follows: (1) System 1 generates multiple metacognitive action nodes, and the process reward model evaluates the node scores, among which the node with the highest reward score is input into System 2; (2) System 2 generates multiple reasoning action nodes, and the process reward model evaluates the node scores; (3) System 1 continues to generate multiple metacognitive action nodes until System 1 evaluates and outputs the correct answer.

7. A novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 6, characterized in that: The interaction between System 1 and System 2 is modeled by the Markov decision process in reinforcement learning. The Markov decision process is defined as a five-tuple as follows: M=(S,A,P,R,γ); Where S is the state space, which represents the metacognitive state or specific problem state in the reasoning process; A is the action space, which represents the action of System 1 or System 2; P is the state transition probability, P(s'|s,a) represents the probability of transitioning to state s' after executing action a in state s; R is the reward function, R(s,a) represents the reward after executing action a in state s; γ is the discount factor used to control the importance of future rewards, 0≤γ≤1; Therefore, the reasoning process performed alternately by System 1 and System 2 can be regarded as a recursive policy problem, where the goal is to maximize the cumulative reward.

8. A novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 7, characterized in that: Based on the reinforcement learning strategy model, the recursive strategy problems of System 1 and System 2 are optimized. The specific process is as follows: Step S321, system 1 and system 2 define strategies as π1(a|s) and π2(a|s), which represent the selection probabilities of metacognitive actions and specific reasoning actions respectively. The optimization goal is to maximize the cumulative reward G t , as shown below: G t =∑ ∞ c k R(s t+k ,a t+k ); Among them, R(s t+k ,a t+k ) is the reward for each step; γ is the discount factor; k represents the step length; Therefore, policy optimization is performed through the policy gradient method as follows: Among them, τ represents the reasoning trajectory; π represents the policy model; θ is the parameter; Step S322: Based on the ActorCritic method, the Actor-Critic framework is adopted to stabilize the strategy optimization process; Actor's strategy for generating actions π θ (a|s); Critic is used to estimate the state value function V(s), that is, to evaluate the current strategy; where the Critic's state value function V(s) represents the expected cumulative reward in state s, as shown below: The TD error is used to update the Critic as follows: δ t =R(s t ,a t )+γV(s t+1 )-V(s t ); Therefore, the Actor's strategy update formula is as follows:

9. A novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 8, characterized in that: In the hierarchical reinforcement learning of System 1 and System 2, each generates actions and optimizes them alternately. The specific process is as follows: Step S331, the reward function R1 of system 1 reflects the contribution of metacognitive actions to the reasoning task, as shown below: R1=f(l,p,q); Among them, l is the problem understanding quality; p is the planning quality; q is the abstract knowledge call quality; The estimation is done through the process reward model as follows: R1=α1·Q l +α2·Q p +α3·Q q ; Among them, α1, α2, and α3 are Q l , Q p and Q q The weight of In step S332, the reward function R2 of system 2 reflects the quality of the specific reasoning step, as shown below: R2=f(t,x,b); Among them, t is the correctness of reasoning; x is efficiency; b is the clarity of steps; In mathematical reasoning tasks: R2=β1·Q t +β2·Q x +β3·Q b ; Among them, β1, β2, and β3 are Q t , Q x and Q b The weight of In step S333, during the interaction between system 1 and system 2, system 1 outputs a metacognitive action a1, and system 2 generates a specific reasoning action a2 based on a1, ultimately forming a complete reasoning trajectory; therefore, the total reward is as follows: R total =R1+R2; The strategy update formula is as follows:

10. A novel large-scale autonomous reasoning system based on fast and slow thinking according to claim 9, characterized in that: The process reward model is used to evaluate the node score Q(s,a) as follows: In the interaction between System 1 and System 2, the reward model is defined as follows: System 1 rewards: scores based on the abstract quality of module combinations; System 2 rewards: scores based on the correctness of specific reasoning steps; Q is iteratively updated using the Bellman equation as follows:

Images