Intelligent agent team strategy processing method and device

Abstract

The invention provides an agent team strategy processing method and device, electronic equipment, a non-instantaneous computer readable storage medium and a computer program product. The method comprises the following steps: constructing an agent team; obtaining the state of each agent and the relationship between the agents; calculating the energy of the agent team according to the state of each agent and the relationship between the agents; acquiring a set of agent states corresponding to each local minimum value according to the energy of the agent team, and performing distributed storage; wherein each agent state set corresponds to one team strategy, the state of each agent in the agent state set serves as a key in the agent local dictionary, and the value of the key represents a task needing to be executed by the agent in the team strategy. According to the invention, the problem that the team strategy of the agent team cannot be effectively prevented from being leaked in the prior art is solved, so that the information security of the team strategy is ensured.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, electronic device, non-transitory computer-readable storage medium, and computer program product for processing intelligent agent team strategies. Background Technology

[0002] An intelligent agent team is a collection of multiple intelligent agents, each possessing a degree of autonomy, decision-making ability, and collaborative capability. These agents can be independent entities or computer programs distributed across different locations. They collaborate to complete specific tasks or achieve a particular goal.

[0003] Adversarial scenarios refer to situations where two competing teams of intelligent agents engage in combat. Given a specific task, the team strategy is the foundation for efficient collaboration and task execution, decisively impacting the team's task outcome. If the team strategy is leaked, the adversary can predict the team's strategy and launch targeted countermeasures, leading to task failure. Current technologies employ complex encryption algorithms to prevent strategy leakage, while simultaneously ensuring that no agent within the team leaks information. This demands extremely high system reliability and security, making implementation extremely difficult.

[0004] There is currently no effective solution to the problem that existing technologies cannot effectively prevent the leakage of team strategies of intelligent agents. Summary of the Invention

[0005] In view of this, the present invention provides a method, apparatus, electronic device, non-transitory computer-readable storage medium, and computer program product for processing intelligent agent team strategies, in order to solve the problem that the prior art cannot effectively prevent the leakage of team strategies of intelligent agent teams.

[0006] To address the aforementioned technical problems, embodiments of the present invention provide a method for processing intelligent agent team strategies, the method comprising:

[0007] Construct an intelligent agent team; wherein the intelligent agent team comprises multiple intelligent agents;

[0008] Obtain the state of each agent and the relationships between them;

[0009] The energy of the agent team is calculated based on the state of each agent and the relationships between the agents.

[0010] The set of agent states corresponding to each local minimum value is obtained based on the energy of the agent team and stored in a distributed manner; wherein, each set of agent states corresponds to a team policy, and the state of each agent in the set of agent states serves as a key in the agent's local dictionary, and the value of the key represents the task that the agent needs to perform in the team policy.

[0011] Optionally, the method further includes:

[0012] The energy of the agent team is calculated using the following formula:

[0013] *E(u)+=F(*u i +,{W ij});

[0014] Among them, u i W represents the state of agent i. ij Let F represent the relationship between agent i and agent j, F represent the preset function, and E(u) represent the energy of the agent team.

[0015] Randomly initialize u i According to the preset W ij Adjust u using a predetermined algorithm i This allows us to find the local minimum of the energy E(u) of the agent team, and thus obtain the set of states for each agent.

[0016] Each agent i according to Extract the tasks to be performed from the local dictionary.

[0017] Optionally, the preset function F and the state u of agent i are used to determine the optimal parameters. i The relationship between agent i and agent j W ij The formula for calculating the energy E(u) of the intelligent agent team is:

[0018]

[0019] Where ρ(·) represents the activation function, b i This represents the amount of bias in agent i;

[0020] Get the preset W ij and b i ;

[0021] Randomly initialize u i According to the preset W ij and b i Adjust u using the predetermined algorithm i We obtain the local minimum of the energy E(u) of the agent team.

[0022] Optionally, obtain the preset W. ij and b i include:

[0023] For any Then E(u) satisfies If the gradient is 0, the following equation holds true, and W can be calculated using the following equation. ij and b i :

[0024]

[0025] in, and W is a given known number. ij and b i It is the unknown to be solved; set The number is M; where M is greater than or equal to a number sufficient to solve W. ij and b i The quantity.

[0026] Optionally, the method further includes:

[0027] Randomly initialize the state u of agent i. i ;

[0028] Agent i sends the current state activation value ρ(u) to agent j. i Or send the current W to agent j. ij ρ(u i );

[0029] Agent i receives ρ(u) from agent j j Or receive W from agent j ij ρ(u j );

[0030] Calculate its own state u according to the following formula. i The gradient until the state u is reached. i The gradient is 0, thus causing the energy E(u) of the intelligent team to reach a local minimum:

[0031] .

[0033] Optionally, the method further includes:

[0034] Agent i receives communication information from agent j; wherein, the communication information includes perturbation information, the perturbation information being obtained by agent j based on W. ij The seed value of the random number generator is calculated by the seed generation algorithm and then generated by the random number generator.

[0035] Agent i calculates the state u of coordinating agent j based on the communication information. j ;

[0036] Based on the state u of agent i in this coordination, j Calculate its own state u i The gradient is calculated and updated.

[0037] Optionally, the method further includes:

[0038] Let G denote the seed generation algorithm, RNG denote the random number generator, and t denote the number of communications between agent i and agent j:

[0039] At t=0, agent i calculates the seed value G(W) shared with agent j. ij ), and use the seed value G(W) ij Initialize the random number generator RNG to RNG. ij ;

[0040] When t≥0, agent i uses a random number generator to generate a perturbation message. And send the perturbed state information to agent j.

[0041] When t≥0, agent i receives the state information after being perturbed by agent j. intelligent agent i according to W ij calculate And according to Further obtain based on Intelligent agent i computation The gradient is updated to

[0042] This invention also provides an intelligent agent team strategy processing device, the device comprising:

[0043] A building module is used to build an intelligent agent team; wherein, the intelligent agent team includes multiple intelligent agents;

[0044] The acquisition module is used to acquire the state of each agent and the relationships between them.

[0045] The first calculation module is used to calculate the energy of the agent team based on the state of each agent and the relationship between each agent.

[0046] The processing module is used to obtain a set of agent states corresponding to each local minimum based on the energy of the agent team, and to perform distributed storage; wherein, each set of agent states corresponds to a team strategy, and the state of each agent in the set of agent states serves as a key in the agent's local dictionary, and the value of the key represents the task that the agent needs to perform in the team strategy.

[0047] Optionally, the device further includes:

[0048] The second calculation module is used to calculate the energy of the agent team using the following formula:

[0049] *E(u)+=F(*u i +,{W ij});

[0050] Among them, u i W represents the state of agent i. ij Let F represent the relationship between agent i and agent j, F represent the preset function, and E(u) represent the energy of the agent team.

[0051] Adjustment module, used for random initialization of u i According to the preset W ij Adjust u using a predetermined algorithm i This allows us to find the local minimum of the energy E(u) of the agent team, and thus obtain the set of states for each agent.

[0052] The execution module is used to control each agent i according to... Extract the tasks to be performed from the local dictionary.

[0053] Optionally, the second calculation module is further configured to calculate the state u of agent i based on the preset function F and the state u of agent i using the following formula. i The relationship between agent i and agent j W ij The energy E(u) of the computational agent team:

[0054]

[0055] Where ρ(·) represents the activation function, b i This represents the amount of bias in agent i;

[0056] The adjustment module is also used to obtain a preset W. ij and b i Randomly initialize u i According to the preset W ij and b i Adjust u using the predetermined algorithm iWe obtain the local minimum of the energy E(u) of the agent team.

[0057] This invention also provides an electronic device, comprising:

[0058] Processor; and

[0059] Stored program memory,

[0060] The program includes instructions that, when executed by the processor, cause the processor to perform an agent team strategy processing method according to any of the above embodiments.

[0061] This invention also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute an agent team strategy processing method in any of the above embodiments.

[0062] This invention also provides a computer program product, which includes instructions that, when executed, cause a computer to perform an intelligent agent team strategy processing method according to any of the above embodiments.

[0063] This invention constructs an intelligent agent team, comprising multiple intelligent agents. It acquires the states of each agent and the relationships between them; calculates the team's energy based on these states and relationships; and obtains a set of agent states corresponding to each local minimum based on the team's energy, storing this set in a distributed manner. Each set of agent states corresponds to a team strategy, and the state of each agent in the set serves as a key in the agent's local dictionary, with the key's value representing the task the agent needs to perform within the team strategy. This solves the problem in existing technologies where the team strategy of an intelligent agent team cannot be effectively prevented from being leaked, thus ensuring the information security of the team strategy.

[0064] It should be understood that the above general description and the following specific embodiments are merely exemplary and illustrative, and do not limit the scope of the invention. Attached Figure Description

[0065] The accompanying drawings, which are part of the specification of this invention, illustrate exemplary embodiments of the invention. The drawings, together with the description in the specification, serve to illustrate the principles of the invention.

[0066] Figure 1 This is a flowchart of an intelligent agent team strategy processing method according to an embodiment of the present invention.

[0067] Figure 2 This is a schematic diagram of an intelligent agent team according to an embodiment of the present invention.

[0068] Figure 3 This is a flowchart of an intelligent agent team strategy processing method according to another embodiment of the present invention.

[0069] Figure 4 This is a structural block diagram of an intelligent agent team strategy processing device according to an embodiment of the present invention.

[0070] Figure 5 This is a structural block diagram of an intelligent agent team strategy processing device according to another embodiment of the present invention.

[0071] Figure 6 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the spirit of the contents disclosed in the present invention will be clearly explained below with reference to the accompanying drawings and detailed description. After understanding the embodiments of the present invention, any person skilled in the art can make changes and modifications based on the technology taught in the present invention without departing from the spirit and scope of the present invention.

[0073] The illustrative embodiments and descriptions of the present invention are used to explain the invention, but are not intended to limit the invention. Furthermore, elements / components using the same or similar reference numerals in the drawings and embodiments are used to represent the same or similar parts.

[0074] The terms "first," "second," etc., used in this document are not intended to specifically refer to order or sequence, nor are they intended to limit the invention. They are merely used to distinguish elements or operations described using the same technical terms.

[0075] The directional terms used in this article, such as up, down, left, right, front, or back, are for reference only when referring to the accompanying drawings. Therefore, the use of directional terms is for illustrative purposes and not to limit this work.

[0076] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0077] The term "and / or" as used herein includes any or all of the things mentioned.

[0078] The term "multiple" in this article includes "two" and "more than two"; the term "multiple groups" in this article includes "two groups" and "more than two groups".

[0079] The terms "approximately," "about," etc., used herein are intended to modify any quantity or error that may vary slightly, but these slight variations or errors do not change the essence of the quantity or error. Generally, the range of slight variations or errors modified by such terms may be 20% in some embodiments, 10% in others, 5% in still others, or other values. Those skilled in the art should understand that the aforementioned values ​​can be adjusted according to actual needs and are not limited thereto.

[0080] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0081] When expressions such as "at least one of A, B, and C" are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). When expressions such as "at least one of A, B, or C" are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., "a system having at least one of A, B, or C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). Those skilled in the art should also understand that any conjunction and / or phrase that substantially arbitrarily indicates two or more optional items, whether in the specification, claims, or drawings, should be understood to indicate the possibility of including one of these items, either of these items, or both items. For example, the phrase “A or B” should be understood as including the possibility of “A” or “B”, or “A and B”.

[0082] An adversarial scenario refers to a situation where two teams of intelligent agents with conflicting interests are competing against each other. Given a specific task, the teams typically have multiple team strategies available for adversarial purposes, and the gains of both teams depend on their respective strategy choices. The adversarial scenario addressed in this invention allows agents within the same team to coordinate before the adversarial action to determine the team strategy to be adopted. Such adversarial scenarios are ubiquitous in real life, such as collusion among players in bridge, pre-auction bidding agreements, coordinated fraud in public places, and tactical deployment in warfare. In these scenarios, the cooperating agents lack the time or means to communicate during the task, and therefore must determine their team strategy through pre-coordination. For clarity, this invention refers to one of the two adversarial teams as the intelligent agent team and the other as the intelligent agent team adversary.

[0083] In existing technologies, adversarial team games are used to model the aforementioned adversarial scenarios. In considering pre-coordination efforts, almost all work focuses solely on achieving a correlated team minimax equilibrium. This equilibrium corresponds to the probability distributions of the agent team and its opponent choosing their respective strategies. These two distributions describe the probability of each strategy being selected by the agent team / opponent. By sampling from their respective distributions, the agent team / opponent can choose a specific team strategy. The significance of "equilibrium" lies in the fact that if the team and its opponent adopted other distributions to select strategies, they would obtain lower expected returns in the adversarial process. However, these approaches assume that the team's specific strategy selection is synchronized with all agents within the team, meaning all agents will execute the strategy. They lack specific mechanisms for pre-coordination to synchronize strategy selection, or the existing pre-coordination mechanisms offer poor security for the team's strategy. For example, modeling the strategy distribution as a signal distribution, where each signal corresponds to a team strategy, involves the team sampling a signal during each pre-coordination. Each agent's strategy uses this signal and their own state as input to output the action to be performed. Since the agents use the same signal, this ensures they adopt the same team strategy. This mechanism is vulnerable to signal leakage. If the currently used signal is leaked, adversaries can predict the team's strategy and launch targeted countermeasures. To eliminate signal leakage, the mechanism design needs additional encryption to ensure reliable and confidential signal synchronization, while also ensuring that no agent in the team leaks the signal. This requires extremely high overall system reliability and security.

[0084] This embodiment provides a method for processing intelligent agent team strategies. Figure 1This is a flowchart of an agent team strategy processing method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0085] Step S101: Construct an agent team. This agent team consists of multiple agents.

[0086] Figure 2 This is a schematic diagram of an intelligent agent team according to an embodiment of the present invention, such as... Figure 2 As shown, a team of intelligent agents is modeled using graphs. Figure 2 This represents the relationships between the agents in the team. Each node represents an agent, and the edges between nodes indicate that the agents are adjacent and can communicate. Figure 2 The data of each node i corresponds to the state of an agent i, and the data of the edges correspond to the relationships between agents.

[0087] Step S102: Obtain the state of each agent and the relationship between each agent.

[0088] Optionally, the states of agents and the relationships between agents can be represented by values, vectors, or matrices. In one optional embodiment, the relationships between agents can specifically be represented by the weights of the edges between agents, which can be represented by an N-dimensional matrix. The data of the edges between agents is fixed because this value determines the number of local energy minimums and the set of states of the agents at each local energy minimum. Therefore, the local energy minimums are fixed only when their values ​​are fixed.

[0089] Step S103: Calculate the energy of the agent team based on the state of each agent and the relationships between them.

[0090] In one alternative embodiment, the edge data between agents is fixed, while the agent states are not. Given the edge data between agents, different agent team energies can be calculated for different agent states.

[0091] Step S104: Obtain the set of agent states corresponding to each local minimum based on the energy of the agent team, and store them in a distributed manner. Each set of agent states corresponds to a team policy. The state of each agent in the set of agent states serves as a key in the agent's local dictionary, and the value of the key represents the task that the agent needs to perform in the team policy. Each agent performs the corresponding part of the task so that the entire team executes the team policy corresponding to the current local minimum energy.

[0092] Generally, team energy has multiple local energy minima. This step establishes a correspondence between these local energy minima and team strategies, allowing the storage of multiple team strategies. For example... Figure 2As shown, in one alternative embodiment, a team strategy storage mechanism based on the Hopfield energy model is used to store different team strategies.

[0093] Through the above steps, an agent team consisting of multiple agents is constructed. The energy of the agent team is calculated based on the state of each agent and the relationships between them. The set of agent states corresponding to each local minimum value is obtained based on the energy of the agent team. Different agent team strategies are stored using an energy model, such as the Hopfield energy model. Compared with the existing technology that uses probability distribution to select specific team strategies, the agent team in this optional embodiment can effectively prevent the team strategy from being leaked, ensuring the information security of the team strategy. At the same time, it simplifies the process and reduces costs without the need for complex encryption technology.

[0094] In an alternative embodiment, the energy of the agent team is calculated using the following formula:

[0095] *E(u)+=F(*u i +,{W ij});

[0096] Among them, u i W represents the state of agent i. ij Let represent the relationship between agents i and j, F represent the preset function, and E(u) represent the energy of the agent team. u is randomly initialized. i According to the preset W ij Adjust u using a predetermined algorithm i This allows us to find the local minimum of the energy E(u) of the agent team, and thus obtain the set of states for each agent. Among them, each agent i according to Extract the task to be executed from the local dictionary. The predetermined algorithm can be gradient descent, conjugate gradient, or Newton's method. Gradient descent, in particular, iteratively updates the agent's state, gradually decreasing the energy value, thus quickly approaching the local energy minimum within a finite number of iterations. The predetermined algorithms listed above are merely illustrative; any algorithm that adjusts u can be used. i All algorithms that obtain the local minimum of the energy E(u) of the intelligent agent team are within the scope of this optional embodiment and are not limited thereto.

[0097] This optional embodiment relates to a distributed secure coordination algorithm for pre-coordination of teams in adversarial scenarios. Graph-based modeling allows for distributed computation algorithms to achieve consensus on strategy selection. Since agents randomly initialize their states during each coordination, it is difficult to predict which local minimum the algorithm converges to each time; that is, the agent team's strategy selection is random, making it difficult for adversaries to predict. Compared to existing technologies, pre-coordination is achieved through local communication and distributed computation, eliminating the need for complex encryption algorithms. Furthermore, even if some agents in the agent team leak their own information, the information of other agents remains confidential, thus ensuring that adversaries cannot obtain global information to infer the agent team's chosen strategy.

[0098] Regarding obtaining the local minimum of the energy E(u) of the agent team, this optional embodiment provides another implementation method. Figure 3 This is a flowchart of an agent team strategy processing method according to another embodiment of the present invention, such as... Figure 3 As shown, it includes the following steps:

[0099] Step S301: Based on the above-mentioned preset function F and the state u of agent i... i The relationship between agent i and agent j W ij The formula for calculating the energy E(u) of the intelligent agent team is:

[0100]

[0101] Where ρ(·) represents the activation function, b i b represents the bias of agent i. i This can be the relationship between agent i and itself, i.e., W ii .

[0102] Step S302: Obtain the preset W ij and b i .

[0103] Step S303: Randomly initialize u i According to the preset W ij and b i Adjust u using a predetermined algorithm i We obtain the local minimum of the energy E(u) of the agent team.

[0104] This optional embodiment provides a convenient coordination mechanism, specifically, given *W ij + and *b i +, Randomly initialize *u i +, the minimum value of the overall energy E(u) can be found through a predetermined algorithm. Using This represents the set of states of each agent when E(u) reaches its minimum value. Since the states of each agent reach... It won't change after that, you can use it. This represents the result of prior coordination by the intelligent agent. It is worth noting that the minimum value of E(u) obtained is usually a local minimum, and energy models generally have multiple local minima.

[0105] In the energy model of the above embodiment, the number of local energy minima of the team energy and the set of states of the agent at the energy minimum point are considered. By *W ij + and *b i +Determine. Therefore, if a randomly generated *W is used... ij + and *b i + It is difficult to determine the number of all local energy minima in the energy model and the set of agent states at the energy minimum point. This affects the keys used to build the agent's local dictionary. To solve this problem, set M keys... As a key in the agent's local dictionary, and calculate *W ij + and *b i + Ensure these The minimum value corresponding to the energy model. In an alternative embodiment, given a Since it requires a corresponding local energy minimum, it must satisfy E(u) in The property that the gradient is 0 is:

[0106]

[0107]

[0108] In the above equation and W is a given known number. ij and b i It is the unknown that is required. Through a... The equations listed are insufficient to determine W. ij and b i Therefore, it was set. The quantity M is large enough to solve for *W ij + and *b i Simply add +.

[0109] Through this optional embodiment, the energy model will not contain unknown local energy minima, effectively preventing the agent's state at energy minima from being unsearchable in the local dictionary. Furthermore, since the number of energy minima is manually set, it ensures that all team strategies can be stored. If a random *W is used... ij + and *bi +, the number of minimum energy values ​​may be less than the number of team strategies.

[0110] In the calculated *W ij + and *b i In the defined energy model, the agent states *u in the agent team i +It can always be stabilized at a certain artificially set value using the gradient descent algorithm. Up. Let each A corresponding team strategy is established and stored in a dictionary maintained locally by the agent: given a Insert a new key into agent i's dictionary The value of this key is The corresponding team strategy requires agent i to perform certain actions. Through this design, each time the agent's state stabilizes at a certain... After that, agent i can then base its actions on the current situation. It retrieves the action it needs to perform from its local dictionary. When all agents select their actions in this way, the team as a whole is executing... Corresponding team strategy.

[0111] Based on the above energy model and team strategy storage mechanism, the pre-coordination process is the process by which each agent adjusts its own state to minimize E(u), and this process can be carried out in a distributed manner. In an optional embodiment, agent i adjusts its own state u i The gradient can be calculated as:

[0112]

[0113] Based on this formula, it can be seen that agent i only needs to know the W corresponding to each of its neighboring agents j. ij ρ(u j The value of ) can be used to calculate its own state u. i The gradient. Agent i locally stores *W related to itself. ij Under the premise of +, pre-coordination can be carried out in a distributed manner, which can be achieved through the following steps: (1) Agent i randomly initializes its own state u i (2) Agent i sends the current state activation value ρ(u) to agent j. i Or send the current W to agent j. ij ρ(u i (3) Agent i receives ρ(u) from agent j. j Or receive W from agent j ij ρ(u j After that, calculate its own state u according to the following formula. iThe gradient is obtained, and the agent repeats steps (2) and (3) until its own state u is obtained. i The gradient is 0, thus causing the energy E(u) of the agent team to reach a local minimum:

[0114]

[0115] The distributed mechanism described in the above embodiments can achieve prior coordination, but if the agent directly sends ρ(u)... i In the case of this, the adversary team of intelligent agents may use this information to calculate u i and u i Gradients and constructing W ij b i The relevant equations. When the opposing intelligent agent team constructs enough equations based on a large amount of information, it can calculate W. ij and b i The values. Having obtained these values, the adversary in the agent team can calculate the goals achieved by the agents in each coordination based on the agents' communication information. This leads to the discovery of the strategy chosen by the agent team, resulting in the leakage of the agent team's strategy.

[0116] To enhance communication security, in one optional embodiment, agents i and j use the same random number generator and seed generation algorithm. The seed generation algorithm can be any algorithm capable of transforming the relationship between agents i and j into a seed value; this optional embodiment does not limit this. Agents i and j can, according to W... ij The seed generation algorithm calculates the seed value of the random number generator and generates the same perturbation information through the random number generator. Agent i receives communication information including the perturbation information from agent j. Agent i is based on W. ij The seed generation algorithm and random number generator obtain perturbation information and calculate the state u of the coordinating agent j. j Agent i, based on the state u of agent j in this coordination... j Calculate its own state u i The gradient is calculated and updated. Specifically, in each coordination, coordination with secure communication includes the following four basic steps:

[0117] (1) Agent i calculates the seed value of the random number generator shared with its neighboring agent j. Before the first communication between agent i and agent j, agent i calculates the seed value based on the relationship W between them. ij A seed value is generated using a seed generation algorithm, and the random number generator is initialized with this seed value. The seed value only needs to be calculated once; the random number generator can continuously generate random numbers using the given seed value. Since agents i and j share W... ijThe seed value calculated by the seed generation algorithm is the same. This ensures that their random number generators will generate the same random numbers. This random number can be an N-dimensional vector used as perturbation information for communication.

[0118] (2) Information perturbation: Each time agent i sends information to agent j, a random number generator is used to generate a random number as the perturbation amount of agent i's current state information. Agent i uses W to perturb the perturbed state information. ij After processing, it will be sent.

[0119] (3) Information De-disturbance: When agent i receives information from agent j, since the random number generators of agent i and agent j will generate the same disturbance information, it can remove the disturbance amount in the state information of agent j according to the disturbance amount of its own current state information, and obtain the current state information of agent j.

[0120] (4) Gradient calculation and state update: Agent i calculates the gradient of its own state based on the state information of agent j after perturbation and updates its own state.

[0121] Add perturbations to the communication information to ensure that when the result of the current coordination is the same as the result of a previous coordination (i.e., ... While the external communication information remains different (the information is the same), this effectively prevents adversaries from memorizing the correspondence between information and team strategy based on the repetition of information. Simultaneously, it ensures that even if the edge weight matrix W of an agent is leaked, the information of agents not directly connected to that agent remains secure, greatly improving system security. This is because the information sent by agents not directly connected to the leaked agent contains two unknown components: the weight matrix W of the agent and its state information. This makes it impossible to calculate the state information of agents not directly connected to the leaked agent, even if the state information of the agents connected to the leaked agent is known.

[0122] In a more specific alternative embodiment, to ensure that the adversary of the intelligent agent team cannot calculate W based on the communication information ij and b i To adjust the communication information between agents, this embodiment of the invention proposes a secure communication mechanism. Specifically, let G represent the seed generation algorithm, RNG represent the random number generator, and t represent the number of communications between agent i and agent j.

[0123] 1) Before agent i communicates with agent j for the first time (i.e., t=0), it calculates the seed value G(W) shared with agent j. ij And use this seed to initialize the random number generator RNG to RNG. ij .

[0124] 2) During the t-th communication, agent i uses a random number generator to generate a perturbation message. And send the perturbed state information to agent j.

[0125] 3) During the t-th communication, agent i receives information from agent j. intelligent agent i according to W ij calculate And according to Further obtain based on Intelligent agent i computation The gradient and update it as

[0126] Through this optional embodiment, since the adversary of the intelligent agent team does not know W ij The value it cannot obtain from Recovery The information. Simultaneously, the random number generator produces random numbers during each communication. They are all different, which means that even if the agent coordinates to the same goal in multiple prior coordinations... Above, what the team's opponents received This is also different, which effectively prevents adversaries from memorizing the correspondence between information and team strategies based on the repetition of information. On the other hand, this optional embodiment also ensures that when the edge weight matrix W of an agent is leaked, the information of agents not directly connected to that agent remains safe, which greatly improves the security of the system. A brief proof of this property is as follows: Figure 2 Taking agents i, j, and k as an example, let's assume that agent k has W ik In the event of a leak, the adversary of the intelligent agent team can access W. ik Calculate the seed value G(W) shared by agents i and k. ik And initialize the random number generator RNG. ik Through RNG ik The intelligent agent team can know And according to the received and Calculated and But it is unattainable. This is because the information that the adversary of the agent team can obtain regarding agent j is and Of this information, the adversary in the intelligent agent team only knows... Therefore, it cannot calculate W. ij and Ensure that even if the communication content of the agents is fully obtained by the team's adversary, the adversary still cannot determine the specific collaboration strategy chosen by the team through prior coordination.

[0127] Instead of traditional probability distributions, the Hopfield energy model is used to store different team strategies. Based on the structure and operational rules of the energy model, a distributed pre-coordination mechanism is further designed. Each agent in the team only needs to communicate with its neighboring agents and perform local computations to achieve consensus on the selection of the collaborative strategy. This mechanism distributes the team strategy information across each agent, ensuring that the leakage of information from some agents does not lead to the leakage of global information. A secure communication mechanism between agents is proposed to ensure that even if the communication content of the agents is fully obtained by the team's adversary, the adversary still cannot determine the specific collaborative strategy selected by the team through pre-coordination.

[0128] This embodiment also provides an intelligent agent team strategy processing device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0129] This embodiment provides an intelligent agent team strategy processing device. Figure 4 This is a structural block diagram of an intelligent agent team strategy processing device according to an embodiment of the present invention, such as... Figure 4 As shown, it includes:

[0130] Module 41 is used to build an agent team; wherein the agent team includes multiple agents;

[0131] The acquisition module 42 is used to acquire the state of each agent and the relationship between each agent;

[0132] The first calculation module 43 is used to calculate the energy of the intelligent agent team based on the state of each intelligent agent and the relationship between each intelligent agent.

[0133] The processing module 44 is used to obtain a set of agent states corresponding to each local minimum based on the energy of the agent team, and to perform distributed storage; wherein, each set of agent states corresponds to a team strategy, and the state of each agent in the set of agent states serves as a key in the agent's local dictionary, and the value of the key represents the task that the agent needs to perform in the team strategy.

[0134] Figure 5 This is a structural block diagram of the processing module according to an embodiment of the present invention, such as... Figure 5 As shown, the intelligent agent team strategy processing device also includes:

[0135] The second calculation module 51 is used to calculate the energy of the agent team using the following formula:

[0136] *E(u)+=F(*u i +,{W ij});

[0137] Among them, u i W represents the state of agent i. ij Let F represent the relationship between agent i and agent j, F represent the preset function, and E(u) represent the energy of the agent team.

[0138] Adjustment module 52 is used to randomly initialize u i According to the preset W ij Adjust u using a predetermined algorithm i This allows us to find the local minimum of the energy E(u) of the agent team, and thus obtain the set of states for each agent.

[0139] Execution module 53, used to, according to each agent i Extract the tasks to be performed from the local dictionary.

[0140] Optionally, the second calculation module 51 is further configured to calculate the state u of agent i based on the preset function F and the state u of agent i using the following formula. i The relationship between agent i and agent j W ij The energy E(u) of the computational agent team:

[0141]

[0142] Where ρ(·) represents the activation function, b i This represents the amount of bias in agent i;

[0143] The adjustment module 52 is also used to obtain the preset W. ij and b i Randomly initialize u i According to the preset W ij and b i Adjust u using this predetermined algorithm i We obtain the local minimum of the energy E(u) of the agent team.

[0144] Further functional descriptions of the above modules are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0145] An exemplary embodiment of the present invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of the present invention.

[0146] An exemplary embodiment of the present invention also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0147] An exemplary embodiment of the present invention also provides a computer program product, including a computer program, wherein, when executed by a computer's processor, the computer program is used to cause the computer to perform a method according to an embodiment of the present invention.

[0148] refer to Figure 6 The present invention will now describe a structural block diagram of an electronic device that can serve as a server or client of the present invention, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0149] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0150] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, output unit 607, storage unit 608, and communication unit 609. Input unit 606 can be any type of device capable of inputting information to electronic device 600. Input unit 606 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 607 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 608 may include, but is not limited to, disks and optical discs. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0151] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above. For example, in some embodiments, the music data processing method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. In some embodiments, the computing unit 601 can be configured by any other suitable means (e.g., by means of firmware) to perform the methods according to embodiments of the present invention.

[0152] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0153] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0154] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0155] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0156] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0157] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0158] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for processing team strategies of intelligent agents, characterized in that, The method includes: Construct an intelligent agent team; wherein the intelligent agent team comprises multiple intelligent agents; Obtain the state of each agent and the relationships between them; The energy of the agent team is calculated based on the state of each agent and the relationships between the agents. The set of agent states corresponding to each local minimum value is obtained based on the energy of the agent team and stored in a distributed manner; wherein, each set of agent states corresponds to a team policy, and the state of each agent in the set of agent states serves as a key in the agent's local dictionary, and the value of the key represents the task that the agent needs to perform in the team policy.

2. The agent team strategy processing method according to claim 1, characterized in that, The method further includes: The energy of the agent team is calculated using the following formula: {E(u)}=F({u i },{W ij }); Among them, u i W represents the state of agent i. ij Let F represent the relationship between agent i and agent j, F represent the preset function, and E(u) represent the energy of the agent team. Randomly initialize u i According to the preset W ij Adjust u using a predetermined algorithm i This allows us to find the local minimum of the energy E(u) of the agent team, and thus obtain the set of states for each agent. Each agent i according to Extract the tasks to be performed from the local dictionary.

3. The agent team strategy processing method according to claim 2, characterized in that, Based on the preset function F and the state u of agent i i The relationship between agent i and agent j W ij The formula for calculating the energy E(u) of the intelligent agent team is: Where ρ(·) represents the activation function, b i This represents the amount of bias in agent i; Get the preset W ij and b i ; Randomly initialize u i According to the preset W ij and b i Adjust u using the predetermined algorithm i We obtain the local minimum of the energy E(u) of the agent team.

4. The agent team strategy processing method according to claim 3, characterized in that, Get the preset W ij and b i include: For any Then E(u) satisfies If the gradient is 0, the following equation holds true, and W can be calculated using the following equation. ij and b i : in, and W is a given known number. ij and b i It is the unknown to be solved; set The number is M; where M is greater than or equal to a number sufficient to solve W. ij and b i The quantity.

5. The agent team strategy processing method according to claim 3, characterized in that, The method further includes: Randomly initialize the state u of agent i. i ; Agent i sends the current state activation value ρ(u) to agent j. i Or send the current W to agent j. ij ρ(u i ); Agent i receives ρ(u) from agent j j Or receive W from agent j ij ρ(u j ); Calculate its own state u according to the following formula. i The gradient until the state u is reached. i The gradient is 0, thus causing the energy E(u) of the intelligent team to reach a local minimum:

6. The agent team strategy processing method according to claim 3, characterized in that, The method further includes: Agent i receives communication information from agent j; wherein, the communication information includes perturbation information, the perturbation information being obtained by agent j based on W. ij The seed value of the random number generator is calculated by the seed generation algorithm and then generated by the random number generator. Agent i calculates the state u of coordinating agent j based on the communication information. j ; Based on the state u of agent i in this coordination, j Calculate its own state u i The gradient is calculated and updated.

7. The agent team strategy processing method according to claim 6, characterized in that, The method further includes: Let G denote the seed generation algorithm, RNG denote the random number generator, and t denote the number of communications between agent i and agent j: At t=0, agent i calculates the seed value G(W) shared with agent j. ij ), and use the seed value G(W) ij Initialize the random number generator RNG to RNG. ij ; When t≥0, agent i uses a random number generator to generate a perturbation message. And send the perturbed state information to agent j. When t≥0, agent i receives the state information after being perturbed by agent j. intelligent agent i according to W ij calculate And according to Further obtain based on Intelligent agent i computation The gradient is updated to 8. The agent team strategy processing method according to any one of claims 2-7, characterized in that, The state of the agents and the relationships between them are represented by values, vectors, and matrices; the predetermined algorithm includes gradient descent, conjugate gradient, or Newton's method.

9. A strategy processing device for an intelligent agent team, characterized in that, The device includes: A building module is used to build an intelligent agent team; wherein, the intelligent agent team includes multiple intelligent agents; The acquisition module is used to acquire the state of each agent and the relationships between them. The first calculation module is used to calculate the energy of the agent team based on the state of each agent and the relationship between each agent. The processing module is used to obtain a set of agent states corresponding to each local minimum based on the energy of the agent team, and to perform distributed storage; wherein, each set of agent states corresponds to a team strategy, and the state of each agent in the set of agent states serves as a key in the agent's local dictionary, and the value of the key represents the task that the agent needs to perform in the team strategy.

10. The intelligent agent team strategy processing device according to claim 9, characterized in that, The device further includes: The second calculation module is used to calculate the energy of the agent team using the following formula: {E(u)}=F({u i },{W ij }); Among them, u i W represents the state of agent i. ij Let F represent the relationship between agent i and agent j, F represent the preset function, and E(u) represent the energy of the agent team. Adjustment module, used for random initialization of u i According to the preset W ij Adjust u using a predetermined algorithm i This allows us to find the local minimum of the energy E(u) of the agent team, and thus obtain the set of states for each agent. The execution module is used to control each agent i according to... Extract the tasks to be performed from the local dictionary.

11. The intelligent agent team strategy processing device according to claim 10, characterized in that, The second calculation module is further configured to calculate the state u of agent i based on the preset function F and the state u of agent i using the following formula. i The relationship between agent i and agent j W ij The energy E(u) of the computational agent team: Where ρ(·) represents the activation function, b i This represents the amount of bias in agent i; The adjustment module is also used to obtain a preset W. ij and b i Randomly initialize u i According to the preset W ij and b i Adjust u using the predetermined algorithm i We obtain the local minimum of the energy E(u) of the agent team.

12. An electronic device, characterized in that, include: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-8.

13. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

14. A computer program product, characterized in that, The computer program product includes instructions that, when executed, cause a computer to perform the method of any one of claims 1 to 8.

Images