A multi-robot cooperation system control method based on reputation expectation fusion mechanism
By introducing a reputation expectation fusion mechanism into a multi-robot collaborative system, and utilizing lattice networks and a prisoner's dilemma game model, the robot's strategy is adjusted to achieve the desired reputation and benefits. This solves the problem of the limitation of the influence of a single mechanism in a multi-robot collaborative system, and improves the efficiency and stability of task execution.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-18
- Publication Date
- 2026-03-31
AI Technical Summary
Existing research focuses only on the impact of a single mechanism on multi-robot collaborative systems, ignoring the complexity and diversity of multiple mechanisms working together. This results in robot decision-making being influenced by multiple factors, and a lack of effective methods to promote cooperative behavior.
A multi-robot collaborative system control method based on reputation expectation fusion mechanism is adopted. By setting up a lattice network or BA network, the parameters of the Prisoner's Dilemma game model and the reputation expectation model are initialized. Combined with Monte Carlo iteration, the robot strategy is adjusted to achieve the expected reputation and benefits. Network reciprocity and noise interference are introduced to promote cooperative behavior.
It improves the execution efficiency and stability of multi-robot collaborative tasks, promotes the emergence of cooperative behaviors, enhances the robustness of the model, and is applicable to real-world multi-robot collaborative tasks.
Smart Images
Figure CN116880288B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robot cooperative control, and in particular to a control method for a multi-robot cooperative system based on a reputation expectation fusion mechanism. Background Technology
[0002] In the field of robot control, reputation mechanisms and expectation mechanisms also have significant practical application value. They can be applied to multi-robot collaborative tasks to improve overall task execution efficiency and stability. The following is an overview of these two mechanisms in robot control: Reputation Mechanism: In multi-robot collaborative tasks, introducing a reputation mechanism helps robots identify well-performing partners. By leveraging reputation to scale robot gains, influence robot strategy choices, and select partners, cooperative behavior can be promoted in multi-robot collaborative systems, which helps improve task completion speed and overall system efficiency. Expectation Mechanism: Expectation mechanisms can incentivize robots to pursue higher gains and better strategies in collaborative tasks. By setting expected gain values for robots, they are prompted to make changes to achieve the desired outcome, thereby improving the level of cooperation in multi-robot collaborative systems. Existing research focuses only on the impact of a single mechanism on multi-robot collaborative systems, neglecting the combined effect of multiple mechanisms. The complexity and diversity of multi-robot collaborative systems mean that each robot's decisions are often influenced by multiple factors, rather than a single mechanism. Summary of the Invention
[0003] To address the problems existing in the prior art, this invention discloses a control method for a multi-robot collaborative system based on a reputation expectation fusion mechanism, specifically including the following steps:
[0004] A network structure is set up for the robot swarm, which can be a lattice network or a BA network, and the robots perform collaborative tasks based on this network structure.
[0005] Initialize the parameters of the Prisoner's Dilemma game model, including the payoff matrix M, the initial cooperative robot nodes, and the Fermi update rule parameters K;
[0006] Initialize the reputation expectation model parameters, including robot reputation value, robot reputation expectation, robot policy duration, and robot reputation sensitivity;
[0007] Based on the Prisoner's Dilemma game model, the robot swarm begins Monte Carlo iteration. During the iteration process, the robot's strategy directly affects the changes in its reputation and payoff. The robot adjusts its own strategy duration or updates its own strategy according to the Fermi rule based on the difference between its individual reputation and reputation expectation, and the difference between its individual payoff and the average payoff of its neighbors, in order to achieve the goal of desired reputation.
[0008] Furthermore, a corresponding network structure model is generated based on the network structure, and the robot swarm is mapped according to the generated network structure, giving it a spatial structure and introducing network reciprocity to limit the interaction range of the robots.
[0009] Furthermore, the payout matrix is initialized by selecting an appropriate temptation T for betrayal, setting the cooperation payout R to 1, and setting the betrayal penalty P and the defrauded payment S to 0.
[0010] A certain number of individual robots are randomly selected as initial cooperative robot nodes, and the initial strategy of these robots is defined as cooperation, while the initial strategy of the remaining robots is defined as betrayal.
[0011] Setting the Fermi update rule parameter K to 0.1 introduces noise interference during policy imitation, indicating that individuals are not perfectly rational.
[0012] Furthermore, for the robot reputation value, the robot reputation range is limited to [0, 30], and the initial reputation value distribution satisfies the uniform distribution of U(0, 30), that is, the average reputation value of the group is 15;
[0013] A suitable expected value is selected as the average reputation expectation of the group, and a normal distribution is used to set the individual reputation expectation of the robot; a suitable value is selected as the average reputation sensitivity of the group, and a normal distribution with suitable parameters is selected to set the individual reputation sensitivity of the robot.
[0014] Furthermore, a single individual is randomly selected from the robot swarm, and this individual engages in a round of game interaction with all its neighbors in the interaction network structure. The payoff for this round is obtained based on the payoff matrix. This process is repeated for each individual in the robot swarm.
[0015] After the entire group completes one round of the game, the strategy duration τ of each robot is reduced by 1;
[0016] The reputation value of each individual is modified according to the strategies of each robot in this round. When a robot chooses the cooperation strategy in this round, the reputation value of that individual increases by 1; when a robot chooses the betrayal strategy in this round, the reputation value of that individual decreases by 1.
[0017] In the robot swarm, a random individual is selected. If the individual's policy duration is 0, it imitates the policy of one of its neighbors with a certain probability according to the Fermi update rule and initializes its policy duration. If the individual's policy duration is not 0, the policy duration τ is adjusted according to the difference between the individual's reputation and the pursuit value, and the difference between the individual's gain and the group's average gain, so that every individual in the robot swarm completes this process.
[0018] Repeat the above steps.
[0019] By employing the aforementioned technical solution, this invention provides a multi-robot cooperative system control method based on a reputation-expectation fusion mechanism. This method proposes a reputation-expectation model (RA model) suitable for multi-robot control by fusing reputation and expectation mechanisms. This model overcomes the limitations of previous studies that focused only on the impact of a single mechanism on cooperative evolution, neglecting the real-world situation where multiple mechanisms work together. Therefore, the RA model is more closely aligned with real-world multi-robot cooperative tasks, improving its robustness. In multi-robot cooperative tasks, the RA model helps promote the emergence of cooperative behaviors, thereby improving task execution efficiency. This model provides new insights for the research and application of robot group game behavior and has the potential to further promote the development of multi-robot control. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 The flowchart of the method of the present invention Detailed Implementation
[0022] To make the technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention:
[0023] like Figure 1 The control method for a multi-robot collaborative system based on a reputation expectation fusion mechanism, as shown, specifically includes the following steps:
[0024] S1: Set up a network structure for the robot group. The network structure is a lattice network or a BA network. The robots perform collaborative tasks based on this network structure.
[0025] S2: Initialize the parameters of the Prisoner's Dilemma game model, including the payoff matrix M, the initial cooperative robot nodes, and the Fermi update rule parameters K;
[0026] S3: Initialize the reputation expectation model parameters, including robot reputation value, robot reputation expectation, robot policy duration, and robot reputation sensitivity;
[0027] S4: Based on the Prisoner's Dilemma game model, the robot group begins Monte Carlo iteration. During the iteration process, the robot's strategy directly affects the changes in its reputation and payoff. The robot adjusts its own strategy duration or updates its own strategy according to the Fermi rule based on the difference between its individual reputation and reputation expectation, and the difference between its individual payoff and the average payoff of its neighbors, in order to achieve the goal of desired reputation.
[0028] Furthermore, based on the definition of a specific complex network structure, a corresponding network structure model is generated, and the robot group is mapped according to the generated network structure, giving it an interaction space structure and introducing network reciprocity to limit the robot's interaction range.
[0029] Furthermore, S2 specifically adopts the following approach:
[0030] S21: For the initialization of the payoff matrix, select an appropriate size for the temptation to betray T, set the cooperation payoff R to 1, and set the betrayal penalty P and the deception payment S to 0. That is, adopt the "weak prisoner's dilemma game model".
[0031] S22: For the selection of initial cooperative robot nodes, we randomly select 50% of the robot individuals as initial cooperative robot nodes, and define the initial strategy of these robots as "cooperation", while the initial strategy of the remaining robots is defined as "betrayal".
[0032] S23: For the Fermi update rule parameter K, we set it to 0.1 to introduce some noise interference in the policy imitation process, indicating that the individual is not perfectly rational.
[0033] Furthermore, S3 specifically adopts the following approach:
[0034] S31: For the robot's initial reputation value, we limit the robot's reputation range to [0, 30], and the distribution of the initial reputation value satisfies the uniform distribution of U(0, 30), that is, the average reputation value of the group is 15.
[0035] S32: For robot reputation expectations, we select an expected value of an appropriate size as the average reputation expectation of the group, and at the same time use a normal distribution with appropriate parameters to set individual reputation expectations for the robots.
[0036] S33: For the robot policy duration, we initialize it to 15 for each robot.
[0037] S34: For robot reputation sensitivity, we select an appropriate value as the average reputation sensitivity of the group, and at the same time use a normal distribution with appropriate parameters to set the individual reputation sensitivity of the robot.
[0038] Furthermore, S4 specifically adopts the following approach:
[0039] S41: Randomly select an individual from the robot swarm, and have that individual engage in a round of game interaction with all its neighbors in the network structure, and obtain the payout for this round of game based on the payout matrix.
[0040] S42: Ensure that each individual robot in the swarm completes step S41.
[0041] S43: After the entire group completes one round of the game, reduce the strategy duration τ of each robot by 1.
[0042] S44: Modify the reputation value of each individual based on the strategies of each robot in this round. When a robot chooses the cooperation strategy in this round, the individual's reputation value increases by 1; when a robot chooses the betrayal strategy in this round, the individual's reputation value decreases by 1.
[0043] S45: Randomly select an individual from the robot swarm. If the individual's policy duration is 0, then according to the Fermi update rule, imitate the policy of one of its neighbors with a certain probability and initialize its policy duration. If the individual's policy duration is not 0, then adjust the policy duration τ according to the difference between the individual's reputation and the pursuit value, and the difference between the individual's gain and the group's gain.
[0044] S46: Ensure that each individual robot in the swarm completes step S45.
[0045] S47: Repeat the above steps.
[0046] In the field of robot control, our Reputation Expectation Model (RA) is applied to multi-robot collaborative tasks based on the traditional Prisoner's Dilemma (PDG) game model. During the game, robots consider factors such as the impact of their decisions on their individual reputation and the difference between their individual reputation and their ideal reputation, thereby prompting cooperative behavior. The RA model consists of three parts: a robot reputation module, a robot reputation expectation module, and a robot policy duration module.
[0047] Robot Reputation Module: In the RA model, R i R(t) represents the reputation of robot i at time t, and its value varies depending on robot i's strategy at time t. When the robot chooses to cooperate, R... i (t) increases by 1, and vice versa. Reputation value, as a form of group information, can accurately describe the robot's historical behavior and cooperative ability. R i The specific change process of (t) is shown in formula (1):
[0048]
[0049] Where D represents the robot adopting a betrayal strategy, C represents the robot adopting a cooperation strategy, Max{A, B} represents taking the maximum value among {A, B}, and Min{A, B} represents taking the minimum value among {A, B}.
[0050] Robot reputation expectation module: In the RA model, R aspi and R asp and represent the reputation expectation values of robot i and the average reputation value of the entire robot group, respectively. The reputation expectation value represents the minimum pursuit of the ideal reputation value by the robot. The difference between the ideal reputation value and the actual reputation value will prompt the robot to adjust its own strategy, which will motivate the robot to maintain a positive attitude in collaborative tasks.
[0051] Robot policy duration module: In the RA model, τ i (t) represents the duration of robot i's policy at time t. The policy duration reflects, to some extent, the robot's preference for its own policy. Meanwhile, θ i Let θ and θ represent the reputation sensitivity of robot i and the average reputation sensitivity of the entire robot group, respectively. Reputation sensitivity represents the strength of a robot's willingness to change its strategy. The change of this parameter occurs in two stages:
[0052] 1. After robot i completes one round of the game, τ i (t) will decrease by 1, indicating that the time robot i adheres to this strategy will decrease by 1. The change process is shown in formula (2):
[0053] τ i (t)=Max{τ i (t-1)-1,0} (2)
[0054] 2. Robot i changes its strategy duration τ by comparing its current reputation with its expected reputation, while also considering the game payoff. i (t). Specifically, when robot i's reputation reaches or exceeds its expected reputation, and its payoff in this round is no less than the average payoff of its neighbors, robot i considers its current strategy effective. Therefore, it increases the strategy duration τ. i (t) and continue with the current strategy; however, if robot i's reputation does not reach the desired reputation, or its payoff in this round is lower than the average payoff of its neighbors, this indicates that robot i's current strategy may not be the optimal choice. In this case, robot i will reduce the strategy duration τ. i (t) to allow for faster experimentation with other strategies. In this way, robot i continuously adjusts the duration of its strategy to seek the optimal strategy in a multi-robot collaborative environment. τ i The specific change process is shown in formula (3):
[0055]
[0056] Where, θ i θ and θ represent the magnitude of robot i's and the group's intention to change their current strategy, respectively. This represents the average profit of robot i at time t. Let represent the average reward of all neighbors of i at time t. Only when τ... i The robot will only choose to imitate its neighbor's strategy and initialize τ when τ decreases to 0. i (t).
[0057] The method proposed in this study consists of two main steps: initialization of the RA model and Monte Carlo iterative (MCS) simulation.
[0058] The RA model initialization process is as follows:
[0059] 1. Initialization of robot reputation module: To ensure the heterogeneity of robot reputation and reflect the different cooperative abilities of different robots in the real world, we use uniform distribution U(0,30) random sampling to complete the initialization of robot reputation.
[0060] 2. Initialization of Robot Reputation Expectation Module: To simulate the different expectations of different robots for cooperative tasks in the real world, we use normal distribution random sampling to initialize the robot reputation expectation.
[0061] 3. Initialization of robot policy duration module: We set the initial policy duration of all robots to 15. At the same time, in order to simulate the magnitude of different robots' willingness to change their own policies in the real world, we used normal distribution random sampling to initialize the robot reputation sensitivity.
[0062] After initializing the RA model, we performed a PDG on an L×L lattice network with periodic boundaries and verified the correctness of the RA model in a multi-robot cooperative system using MCS simulations. The specific process is as follows:
[0063] 1. First, randomly select robot i, let the robot play against all its neighbors and record its own payout.
[0064] 2. After robot i completes this round of the game, the duration τ of strategy i is increased. i (t) decreases by 1.
[0065] 3. Based on the strategy adopted in this round, S i (t) Update its reputation R i (t).
[0066] 4. If the duration of strategy i at this time is τi If (t) is not 0, then i will update its policy duration according to formula (3); if the policy duration τ of i is not 0, then i will update its policy duration according to formula (3); i If (t) is 0, then i will randomly select individual j from its neighbors and decide whether to imitate strategy S according to the Fermi criterion. j (t), after this stage ends, τ will be... i (t) is reset to the initial value. The Fermi criterion is shown in Equation (4), where K represents the uncertainty in the policy imitation stage, and the larger the value of K, the higher the uncertainty.
[0067]
[0068] In multi-robot collaborative tasks, once all robots complete the aforementioned process, one round of synchronous game play is considered complete. Next, the system cooperation rate can be calculated and recorded. After a specified number of rounds of synchronous game play, the cooperation rate recorded during the iteration process is persistently saved to a file. By analyzing the trend of the system cooperation rate in the file, we can find that the introduction of the RA model promotes the emergence of cooperative phenomena in multi-robot collaborative groups. This cooperative phenomenon helps improve the execution efficiency of multi-robot collaborative tasks, thereby achieving more efficient collaborative work in practical applications. The introduction of the RA model provides a new approach to studying the game behavior of robot groups and has the potential to promote further development in the field of multi-robot control.
[0069] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for controlling a multi-robot cooperation system based on a reputation expectation fusion mechanism, characterized in that The application relates to a method for simulating a robot swarm, and belongs to the field of swarm intelligence. A network structure is set for the robot swarm, the network structure is a lattice network or a BA network, and the robots perform a cooperative task based on the network structure; Parameters of a prisoner's dilemma game model are initialized, including a payoff matrix M, initial cooperative robot nodes, and Fermi updating rule parameters K; Parameters of a reputation expectation model are initialized, including robot reputation values, robot reputation expectations, robot strategy durations, and robot reputation sensitivities; Based on the prisoner's dilemma game model, the robot swarm starts to perform Monte Carlo iteration, in the iteration process, the strategy of the robot directly influences changes of the reputation and the payoff of the robot, the robot adjusts the strategy duration according to a difference between the individual reputation and the reputation expectation and a difference between the individual payoff and the average payoff of neighbors, or updates the strategy according to the Fermi rule to achieve the expected reputation target; One individual is randomly selected from the robot swarm, the individual and all neighbors in the interaction network structure of the individual perform one round of game interaction, and the payoff of the one round of game interaction is obtained according to the payoff matrix; each individual in the robot swarm completes the process; After the whole swarm completes one round of game, the strategy duration tau of each robot is reduced by 1; The reputation value information of each individual is modified according to the strategy of each robot in the round, when the robot selects a cooperation strategy in the round, the reputation value of the individual is increased by 1; when the robot selects a betrayal strategy in the round, the reputation value of the individual is reduced by 1; One individual is randomly selected from the robot swarm, if the strategy duration of the individual is 0, the strategy of a certain individual in the neighbors is imitated according to the Fermi updating rule with a certain probability, and the strategy duration of the individual is initialized; if the strategy duration of the individual is not 0, the strategy duration tau is adjusted according to a difference between the individual reputation and the pursuit value and a difference between the individual payoff and the average payoff of the swarm, so that each individual in the robot swarm completes the process. 2.The multi-robot cooperation system control method based on reputation expectation fusion mechanism according to claim 1, characterized in that: A corresponding network structure model is generated according to the network structure, the robot swarm is mapped according to the generated network structure, a spatial structure is given to the robot swarm, network reciprocity is introduced, and the interaction range of the robot is limited.
3. The multi-robot cooperation system control method based on the reputation expectation fusion mechanism according to claim 1, characterized in that: The payoff matrix is initialized, a suitable size of betrayal temptation T is selected, the cooperation payoff R is defined as 1, and the betrayal punishment P and the cheated payment S are defined as 0; A certain number of robot individuals are randomly selected as initial cooperative robot nodes, and the initial strategy of the robot individuals is defined as cooperation, and the initial strategy of the remaining robots is defined as betrayal; For the Fermi updating rule parameters K, the parameters are set as 0.1, noise interference is introduced in the strategy imitation process, and the non-complete rationality of the individual is indicated.
4. The multi-robot cooperation system control method based on the reputation expectation fusion mechanism according to claim 1, characterized in that: The reputation range of the robot is limited to [0, 30], and the initial reputation value is uniformly distributed in U(0, 30), that is, the average reputation value of the swarm is 15; A suitable size of expectation value is selected as the average reputation expectation of the swarm, a normal distribution is used to set the individual reputation expectation of the robot, a suitable size of value is selected as the average reputation sensitivity of the swarm, and a normal distribution with a suitable parameter is used to set the individual reputation sensitivity of the robot.