Reinforcement learning reward and penalty mechanism design method for agent thinking chain

Abstract

The invention discloses a reinforcement learning reward and punishment mechanism design method for an intelligent agent thinking chain, and belongs to the technical field of artificial intelligence, and the method comprises the following steps: S1, state and action definition; s2, sub-task state and action planning; s3, constructing a thinking chain path; s4, a thinking chain step reward and punishment mechanism; and S5, a thinking chain path reward and punishment mechanism. According to the method, each step in the thinking chain can obtain instant reward and punishment so as to accurately guide the decision of each step, and meanwhile, overall evaluation is provided through a reward and punishment mechanism after the task is finished, so that the task execution accuracy can be greatly improved when the intelligent agent faces complex problems.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain. Background Art

[0002] Most current intelligent agents rely on models in natural language processing for language understanding and generation. In conversational intelligent agents, models perform semantic analysis and contextual understanding of language by training on a large corpus, and generate answers based on the input content. When facing complex problems, current intelligent agents use a multi-step reasoning method called "thinking chain". The result of each step of reasoning is used as the input for the next step. The intelligent agent generates a series of state-action pairs based on historical interactions and the context of the problem. The purpose is to simulate the logical reasoning process of human thinking and achieve better performance in complex tasks. With the development of technology, reinforcement learning is being used to train intelligent agents to generate thinking chains. By providing the intelligent agent with an environment and a reference path, the thinking chain path generation strategy is learned through trial and error. Reinforcement learning is mostly based on reward and punishment signals for reward and punishment evaluation. The goal of the intelligent agent is to maximize the reward and punishment signals and gradually optimize its generation strategy, that is, to select the best action to complete the task in a complex task. Many existing systems build intelligent agents based on such algorithms, enabling them to complete specific tasks in a given environment.

[0003] But there are several shortcomings: Traditional intelligent agents have single actions and lack semantic interaction, so it is necessary to define enhanced training of the intelligent agent's thinking chain.

[0004] The reward and punishment mechanism for the intelligent agent's thinking chain is imperfect. The traditional reward and punishment signals rely on the overall evaluation at the end of the task, which makes it difficult to accurately guide the decision-making at each step, and no reward and punishment mechanism is proposed for the thinking chain path planning.

[0005] In existing reinforcement learning and thinking chain construction methods, intelligent agents are often limited to a single solution to a specific problem and lack the ability to explore diverse paths, resulting in obvious limitations when dealing with different situations or variant problems.

[0006] A Chinese patent document with publication number CN118331199A and publication date July 12, 2024 discloses a multi-task collaborative processing method based on artificial intelligence agent and event chain, including the following steps: Step S1, using generative artificial intelligence to build an artificial intelligence agent system to simulate, predict and optimize multi-task autonomous scheduling of complex industrial process scenarios; Step S2: by fusion learning of multi-source and multi-modal data in the industrial process, multiple association models between complex tasks are established, and the comprehensive learning ability of the generative pre-trained model is used to comprehensively model the multi-tasks of the industrial process; Step S3, the artificial intelligence agent system establishes a multi-task event trigger and response mechanism by learning the correlation and interaction between complex tasks in the industrial process, and captures the key event process chain and thinking chain in the industrial process; Step S4: Through the artificial intelligence agent system, the multi-task scenarios of the industrial process are deeply understood to realize the multi-task collaborative processing of complex industrial scenarios.

[0007] The patent document discloses a multi-task collaborative processing method based on artificial intelligence agents and event chains, and proposes a complex scenario multi-task autonomous scheduling framework based on thinking chains. Based on the artificial intelligence agent system, it analyzes the operation requirements of industrial processes and decomposes complex manufacturing tasks into multiple scenario tasks; based on specific scenario industrial data sets and domain expert knowledge training, a scenario-based industrial big model is obtained to support real-time analysis and intelligent decision-making of scenario tasks; the interdependence between multiple tasks is mined, and the time sensitivity, resource requirements and priority factors of multiple tasks are comprehensively considered, which can improve the operation efficiency and resource utilization of industrial processes. However, each step in the thinking chain cannot receive immediate rewards and penalties, and cannot guide the precise decision-making of each step, which affects the accuracy of the intelligent agent in performing tasks when facing complex problems. Summary of the invention

[0008] In order to overcome the defects of the above-mentioned prior art, the present invention provides a method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent's thinking chain. Each step in the thinking chain of the present invention can receive immediate rewards and punishments to accurately guide the decision-making of each step. At the same time, after the task is completed, an overall evaluation is provided through the reward and punishment mechanism, so that the intelligent agent can greatly improve the accuracy of task execution when facing complex problems.

[0009] The present invention is achieved through the following technical solutions: A method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain includes the following steps: S1, state and action definition; Define the agent's current state and executable actions; S2, subtask status and action planning; According to the execution result of the previous subtask, plan the state and action of the next subtask, and use the execution result of each subtask as the input of the next task; S3, construction of thinking chain path; Define a thought chain path, which consists of multiple sets of state-action pairs. The path goes from the initial state to the target state. The agent advances the task through reasoning and decision-making at each step. S4, reward and punishment mechanism for thinking chain steps; Set up a reward and penalty mechanism for each step in the thinking chain path, and calculate the immediate reward and penalty. Evaluate through the multi-dimensional reward and penalty mechanism to calculate the total score of the thinking chain steps; S5, Thinking Chain Path Reward and Penalty Mechanism; Multiple path reward and punishment dimensions are designed to evaluate the path properties, the total score of the thinking chain path properties is calculated, and the total reward and punishment function is obtained by combining the total score of the thinking chain steps.

[0010] In S1, the current state includes user questions , context information Interaction history , current status , executable actions include asking questions ,search ,reasoning and generate answers , executable actions .

[0011] In S2, the state of the next subtask ,in The execution result of the previous subtask; the action of the next subtask ,in Action planning for the next subtask.

[0012] In S3, the thought chain path is defined as: ; in, For the thinking chain path, is the initial state, For the initial action, is the status of the second subtask, is the action of the second subtask, For the end state, The end action of the subtask.

[0013] In S4, setting a reward and penalty mechanism for each step in the thinking chain path refers to setting rewards and penalties for correctness, rewards and penalties for relevance, and rewards and penalties for information density.

[0014] In said S4, the instant reward and penalty are calculated by formula 1; Formula 1; in, For immediate rewards and punishments, is the weight of the correctness reward and penalty, Rewards and penalties for correctness, is the weight of the relevance reward and penalty, Rewards and penalties for relevance, is the weight of coverage bonus and penalty, Rewards and penalties for coverage.

[0015] In S4, the evaluation through a multi-dimensional reward and penalty mechanism refers to evaluating the correctness score of the answer to each step of the thinking chain through correctness rewards and penalties, evaluating the relevance of the action, state and dialogue context of each step of the thinking chain through relevance rewards and penalties, and quantifying the useful information in the answer to each step of the thinking chain through information density rewards and penalties.

[0016] In S4, the total score of the thought chain steps is calculated by formula 2; Formula 2; in, Give a total score for the thought chain steps. is the final time step of the path.

[0017] In S5, the total score of the thought chain path property is calculated by formula 3; Formula 3; in, is the total score of the path nature of the thinking chain, is a linear penalty, Rewards and penalties for path diversity, is the path reconciliation evaluation degree.

[0018] In S5, the total reward and penalty function is calculated by formula 4; Formula 4; in, is the total reward and penalty function.

[0019] The intelligent agent described in the present invention refers to an agent that can perceive the environment and take actions to achieve specific goals, and has autonomy, adaptability and interaction capabilities; the intelligent agent perceives changes in the environment, such as through sensors or data input, makes judgments and decisions based on the knowledge and algorithms it has learned, and then performs actions to affect the environment or achieve predetermined goals.

[0020] The thought chain described in the present invention refers to the process of breaking down a logically complex problem and forming a complete thinking process through a series of logically related thinking.

[0021] The policy network described in the present invention is a neural network whose core function is to output the probability distribution of each possible action taken in this state or directly output a specific action based on the current environmental state input; when processing image data, the front end of the policy network usually includes convolutional layers, which can extract key features of the environmental state.

[0022] The BM25 described in the present invention is an information retrieval algorithm for evaluating the relevance between a document and a query, and the relevance of a document is evaluated by calculating the weighted sum of term frequency and inverse document frequency.

[0023] The beneficial effects of the present invention are mainly manifested in the following aspects: 1. Compared with the prior art, the present invention can provide immediate rewards and penalties for each step in the thinking chain to accurately guide the decision-making of each step. At the same time, after the task is completed, an overall evaluation is provided through the reward and punishment mechanism, so that the intelligent agent can greatly improve the accuracy of task execution when facing complex problems.

[0024] 2. The present invention mathematically defines the existing intelligent agent thinking chain, optimizes the intelligent agent's reasoning path evaluation through a multi-dimensional reward and punishment mechanism, and solves the problem of insufficient reward and punishment mechanism in existing reinforcement learning.

[0025] 3. The present invention guides the intelligent agent to explore more paths through a diverse reward and punishment mechanism, thus avoiding the intelligent agent being limited to a single solution and making the solutions more diversified.

[0026] 4. The present invention can quantify the matching degree of the action, state, and dialogue context of each step of the thought chain through BM25.

[0027] 5. The present invention can quantify the balance of different paths generated by questions and evaluate the coverage and accuracy of key points in the paths. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Figure 1 It is a flowchart of the present invention. DETAILED DESCRIPTION

[0029] Example 1 See also Figure 1 , a method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain, comprising the following steps: S1, state and action definition; Define the agent's current state and executable actions; S2, subtask status and action planning; According to the execution result of the previous subtask, plan the state and action of the next subtask, and use the execution result of each subtask as the input of the next task; S3, construction of thinking chain path; Define a thought chain path, which consists of multiple sets of state-action pairs. The path goes from the initial state to the target state. The agent advances the task through reasoning and decision-making at each step. S4, reward and punishment mechanism for thinking chain steps; Set up a reward and penalty mechanism for each step in the thinking chain path, and calculate the immediate reward and penalty. Evaluate through the multi-dimensional reward and penalty mechanism to calculate the total score of the thinking chain steps; S5, Thinking Chain Path Reward and Penalty Mechanism; Multiple path reward and punishment dimensions are designed to evaluate the path properties, the total score of the thinking chain path properties is calculated, and the total reward and punishment function is obtained by combining the total score of the thinking chain steps.

[0030] This embodiment is the most basic implementation method. Compared with the existing technology, each step in the thinking chain can receive immediate rewards and punishments to accurately guide the decision-making of each step. At the same time, after the task is completed, an overall evaluation is provided through a reward and punishment mechanism, so that the intelligent agent can greatly improve the accuracy of task execution when facing complex problems.

[0031] Example 2 See also Figure 1 , a method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain, comprising the following steps: S1, state and action definition; Define the agent's current state and executable actions; S2, subtask status and action planning; According to the execution result of the previous subtask, plan the state and action of the next subtask, and use the execution result of each subtask as the input of the next task; S3, construction of thinking chain path; Define a thought chain path, which consists of multiple sets of state-action pairs. The path goes from the initial state to the target state. The agent advances the task through reasoning and decision-making at each step. S4, reward and punishment mechanism for thinking chain steps; Set up a reward and penalty mechanism for each step in the thinking chain path, and calculate the immediate reward and penalty. Evaluate through the multi-dimensional reward and penalty mechanism to calculate the total score of the thinking chain steps; S5, Thinking Chain Path Reward and Penalty Mechanism; Multiple path reward and punishment dimensions are designed to evaluate the path properties, the total score of the thinking chain path properties is calculated, and the total reward and punishment function is obtained by combining the total score of the thinking chain steps.

[0032] In S1, the current state includes user questions , context information Interaction history , current status , executable actions include asking questions ,search ,reasoning and generate answers , executable actions .

[0033] In S2, the state of the next subtask ,in The execution result of the previous subtask; the action of the next subtask ,in Action planning for the next subtask.

[0034] This embodiment is a preferred implementation method, which mathematically defines the existing intelligent agent thinking chain, optimizes the intelligent agent's reasoning path evaluation through a multi-dimensional reward and punishment mechanism, and solves the problem of insufficient reward and punishment mechanism in existing reinforcement learning.

[0035] Example 3 See also Figure 1 , a method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain, comprising the following steps: S1, state and action definition; Define the agent's current state and executable actions; S2, subtask status and action planning; According to the execution result of the previous subtask, plan the state and action of the next subtask, and use the execution result of each subtask as the input of the next task; S3, construction of thinking chain path; Define a thought chain path, which consists of multiple sets of state-action pairs. The path goes from the initial state to the target state. The agent advances the task through reasoning and decision-making at each step. S4, reward and punishment mechanism for thinking chain steps; Set up a reward and penalty mechanism for each step in the thinking chain path, and calculate the immediate reward and penalty. Evaluate through the multi-dimensional reward and penalty mechanism to calculate the total score of the thinking chain steps; S5, Thinking Chain Path Reward and Penalty Mechanism; Multiple path reward and punishment dimensions are designed to evaluate the path properties, the total score of the thinking chain path properties is calculated, and the total reward and punishment function is obtained by combining the total score of the thinking chain steps.

[0036] In S1, the current state includes user questions , context information Interaction history , current status , executable actions include asking questions ,search ,reasoning and generate answers , executable actions .

[0037] In S2, the state of the next subtask ,in The execution result of the previous subtask; the action of the next subtask ,in Action planning for the next subtask.

[0038] In S3, the thought chain path is defined as: ; in, For the thinking chain path, is the initial state, For the initial action, is the status of the second subtask, is the action of the second subtask, For the end state, In S4, setting a reward and penalty mechanism for each step in the thought chain path refers to setting a correctness reward and penalty, a relevance reward and penalty, and an information density reward and penalty.

[0039] In said S4, the instant reward and penalty are calculated by formula 1; Formula 1; in, For immediate rewards and punishments, is the weight of the correctness reward and penalty, Rewards and penalties for correctness, is the weight of the relevance reward and penalty, Rewards and penalties for relevance, is the weight of coverage bonus and penalty, Rewards and penalties for coverage.

[0040] In S4, the evaluation through a multi-dimensional reward and penalty mechanism refers to evaluating the correctness score of the answer to each step of the thinking chain through correctness rewards and penalties, evaluating the relevance of the action, state and dialogue context of each step of the thinking chain through relevance rewards and penalties, and quantifying the useful information in the answer to each step of the thinking chain through information density rewards and penalties.

[0041] In S4, the total score of the thought chain steps is calculated by formula 2; Formula 2; in, Give a total score for the thought chain steps. is the final time step of the path.

[0042] This embodiment is another preferred implementation method, which guides the intelligent agent to explore more paths through a diversity reward and penalty mechanism, avoiding the intelligent agent being limited to a single solution, and making the solutions more diverse.

[0043] Example 4 See also Figure 1 , a method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain, comprising the following steps: S1, state and action definition; Define the agent's current state and executable actions; S2, subtask status and action planning; According to the execution result of the previous subtask, plan the state and action of the next subtask, and use the execution result of each subtask as the input of the next task; S3, construction of thinking chain path; Define a thought chain path, which consists of multiple sets of state-action pairs. The path goes from the initial state to the target state. The agent advances the task through reasoning and decision-making at each step. S4, reward and punishment mechanism for thinking chain steps; Set up a reward and penalty mechanism for each step in the thinking chain path, and calculate the immediate reward and penalty. Evaluate through the multi-dimensional reward and penalty mechanism to calculate the total score of the thinking chain steps; S5, Thinking Chain Path Reward and Penalty Mechanism; Multiple path reward and punishment dimensions are designed to evaluate the path properties, the total score of the thinking chain path properties is calculated, and the total reward and punishment function is obtained by combining the total score of the thinking chain steps.

[0044] In S1, the current state includes user questions , context information Interaction history , current status , executable actions include asking questions ,search ,reasoning and generate answers , executable actions .

[0045] In S2, the state of the next subtask ,in The execution result of the previous subtask; the action of the next subtask ,in Action planning for the next subtask.

[0046] In S3, the thought chain path is defined as: ; in, For the thinking chain path, is the initial state, For the initial action, is the status of the second subtask, is the action of the second subtask, For the end state, In S4, setting a reward and penalty mechanism for each step in the thought chain path refers to setting a correctness reward and penalty, a relevance reward and penalty, and an information density reward and penalty.

[0047] In said S4, the instant reward and penalty are calculated by formula 1; Formula 1; in, For immediate rewards and punishments, is the weight of the correctness reward and penalty, Rewards and penalties for correctness, is the weight of the relevance reward and penalty, Rewards and penalties for relevance, is the weight of coverage bonus and penalty, Rewards and penalties for coverage.

[0048] In S4, the evaluation through a multi-dimensional reward and penalty mechanism refers to evaluating the correctness score of the answer to each step of the thinking chain through correctness rewards and penalties, evaluating the relevance of the action, state and dialogue context of each step of the thinking chain through relevance rewards and penalties, and quantifying the useful information in the answer to each step of the thinking chain through information density rewards and penalties.

[0049] In S4, the total score of the thought chain steps is calculated by formula 2; Formula 2; in, Give a total score for the thought chain steps. is the final time step of the path.

[0050] In S5, the total score of the thought chain path property is calculated by formula 3; Formula 3; in, is the total score of the path nature of the thinking chain, is a linear penalty, Rewards and penalties for path diversity, is the path reconciliation evaluation degree.

[0051] In S5, the total reward and penalty function is calculated by formula 4; Formula 4; in, is the total reward and penalty function.

[0052] This embodiment is the best implementation method, which can quantify the matching degree of the action, state, and dialogue context of each step of the thought chain through BM25.

[0053] It can quantify the balance of different paths generated by questions and evaluate the coverage and accuracy of key points in the paths.

[0054] The reward and punishment mechanism for the thinking chain steps includes rewards and punishments for correctness , Relevance Rewards and Penalties and information density rewards and penalties .

[0055] Correctness Rewards and Penalties :If a step of the system gives an incorrect answer or inappropriate reasoning, the score should be negative or zero. By evaluating the correctness of each step, positive and negative rewards and penalties are given: Relevance Rewards and Penalties : The matching metric based method BM25 measures the matching degree between the answer and the question or context.

[0056] BM25 is a ranking function in information retrieval that is used to evaluate the relevance between documents and queries. BM25 scores documents based on word frequency and document length, and is suitable for evaluating the relevance of answers to questions and context.

[0057] in, Words In answer The frequency of words in ; To answer Length; is the average length of all responses and is a constant; To adjust the parameters, The value is 1.2; To adjust the parameters, The value is 0.75; Words The inverse document frequency of .

[0058] in, is the total number of document collections, To contain words The number of documents.

[0059] Information density rewards and penalties : Information density is a measure of the ratio of useful information to total information in a response.

[0060] Information density is measured by measuring the amount of key information in the answer. Defined as a ratio to the total length of the answer: in, is the length of the key information in the answer, The total length of the answer.

[0061] The thinking chain path reward and punishment mechanism includes path diversity reward and punishment , Coverage Rate Rewards and Penalties , Accuracy and path reconciliation evaluation .

[0062] Path diversity rewards and penalties : Path diversity rewards and penalties are used to encourage the system to generate more diverse solutions to problems and avoid falling into local optimal solutions. Highly diverse paths can cover more possible solution spaces, improve the flexibility of task completion and the robustness of the system.

[0063] A thought chain path It is a sequence consisting of a series of subtasks, expressed as: in, The path of thought chain The subtasks at time step t are different action decisions, reasoning steps or problem decompositions, expressed as , is a state-action pair.

[0064] Path diversity definition: for the same problem, set a group of thinking chain paths , each path represents a decision sequence or thinking chain of the system on a certain task. The diversity of paths is reflected in the differences between paths. The diversity of paths is judged by measuring the similarity between different paths. The more dissimilar the two paths are, the higher the diversity. The differences between paths are measured by edit distance or hash distance.

[0065] Coverage bonus and penalty : Coverage bonus and penalty are used to evaluate whether the generated thinking chain path covers all the key points in the problem or whether it meets the key requirements of the task goal. Coverage is calculated by comparing with the predefined reference answer or task goal. Coverage is defined as the degree of overlap between the system output and the target content.

[0066] Accuracy : Precision is used to measure how many relevant key points there are in the content generated by the system; it is defined as the proportion of key points in the content generated by the system that are truly reference answers: in, The number of critical steps included in the intersection of the system-generated path and the reference path, The number of steps in the generated path.

[0067] The precision rate considers the accuracy of the generated steps to avoid the system generating a large number of irrelevant or redundant steps. The precision rate ranges from [0,1], where 1 means that all the generated steps hit the key information, and 0 means that no key information is hit.

[0068] Path Harmony Evaluation :By introducing a weighting coefficient To adjust coverage rewards and penalties And precision The relative importance of: in, is the weighting coefficient; When >1, the weight of coverage bonus and penalty is larger, emphasizing finding more key information; When <1, the weight of precision is larger, emphasizing that the generated path is more accurate and contains less irrelevant information.

[0069] In order to prevent coverage bonuses or extreme accuracy, a penalty function is introduced To dynamically adjust.

[0070] in, is the penalty coefficient, which is used to control the intensity of the penalty; Path Harmony Evaluation .

Claims

1. A method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain, characterized in that: The following steps are involved: S1, state and action definition; Define the agent's current state and executable actions; S2, subtask status and action planning; According to the execution result of the previous subtask, plan the state and action of the next subtask, and use the execution result of each subtask as the input of the next task; S3, construction of thinking chain path; Define a thought chain path, which consists of multiple sets of state-action pairs. The path goes from the initial state to the target state. The agent advances the task through reasoning and decision-making at each step. S4, reward and punishment mechanism for thinking chain steps; Set up a reward and penalty mechanism for each step in the thinking chain path, and calculate the immediate reward and penalty. Evaluate through the multi-dimensional reward and penalty mechanism to calculate the total score of the thinking chain steps; S5, Thinking Chain Path Reward and Penalty Mechanism; Multiple path reward and punishment dimensions are designed to evaluate the path properties, the total score of the thinking chain path properties is calculated, and the total reward and punishment function is obtained by combining the total score of the thinking chain steps.

2. The method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain according to claim 1, characterized in that: In S1, the current state includes user questions , context information Interaction history , current status , executable actions include asking questions ,search ,reasoning and generate answers , executable actions .

3. The method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain according to claim 1, characterized in that: In S2, the state of the next subtask ,in The execution result of the previous subtask; the action of the next subtask ,in Action planning for the next subtask.

4. The method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain according to claim 1, characterized in that: In S3, the thought chain path is defined as: ； in, For the thinking chain path, is the initial state, For the initial action, is the status of the second subtask, is the action of the second subtask, For the end state, The end action of the subtask.

5. The method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain according to claim 1, characterized in that: In S4, setting a reward and penalty mechanism for each step in the thinking chain path refers to setting rewards and penalties for correctness, rewards and penalties for relevance, and rewards and penalties for information density.

6. The method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain according to claim 5, characterized in that: In said S4, the instant reward and penalty are calculated by formula 1; Formula 1; in, For immediate rewards and punishments, is the weight of the correctness reward and penalty, Rewards and penalties for correctness, is the weight of the relevance reward and penalty, Rewards and penalties for relevance, is the weight of coverage bonus and penalty, Rewards and penalties for coverage.

7. The method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain according to claim 5, characterized in that: In S4, the evaluation through a multi-dimensional reward and penalty mechanism refers to evaluating the correctness score of the answer to each step of the thinking chain through correctness rewards and penalties, evaluating the relevance of the action, state and dialogue context of each step of the thinking chain through relevance rewards and penalties, and quantifying the useful information in the answer to each step of the thinking chain through information density rewards and penalties.

8. The method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain according to claim 1, characterized in that: In S4, the total score of the thought chain steps is calculated by formula 2; Formula 2; in, Give a total score for the thought chain steps. is the final time step of the path.

9. The method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain according to claim 8, characterized in that: In S5, the total score of the thought chain path property is calculated by formula 3; Formula 3; in, is the total score of the path nature of the thinking chain, is a linear penalty, Rewards and penalties for path diversity, is the path reconciliation evaluation degree.

10. The method for designing a reinforcement learning reward and punishment mechanism for an intelligent agent thinking chain according to claim 9, characterized in that: In S5, the total reward and penalty function is calculated by formula 4; Formula 4; in, is the total reward and penalty function.

Images