Formation control method, device, equipment and storage medium for intelligent agent network

By determining the agent status and neighbor information, generating an index set, local coordinate system transformation and graph neural network initialization, the problem of difficulty in achieving qualitative and quantitative control of multi-agent formations in the existing technology is solved, and the quantitative of the agent network is realized Formation control and convergence guarantee.

CN118707962BActive Publication Date: 2024-11-01PENG CHENG LAB
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411203645.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-01
Estimated Expiration
2044-08-30

AI Technical Summary

Technical Problem

The existing multi-agent formation control method cannot achieve qualitative and quantitative control, especially in constrained nonlinear handover network systems, and only local information is used, it is difficult to ensure convergence, collision avoidance and connectivity.

Method used

By determining the status information of the agent and the neighbor agent, generating an index set, performing local coordinate system transformation, determining the pursuit and escape micrograph, initializing the graph neural network, and constraining the weight domain, we can achieve the determination of the target formation control information.

Benefits of technology

Qualitative and quantitative formation control of agents in the agent network is realized, ensuring convergence, collision avoidance and connectivity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118707962B_ABST
    Figure CN118707962B_ABST
Patent Text Reader

Abstract

The present application discloses a formation control method, device, equipment and storage medium for an intelligent agent network, and relates to the field of formation control technology. The present application includes: respectively determining the state information of intelligent agents in the intelligent agent network and the neighboring intelligent agents of each intelligent agent, generating an index set according to the neighboring state information of the neighboring intelligent agents, obtaining the formation state based on the index set, uniformly transforming the intelligent agents corresponding to the formation state according to the local coordinate system of the neighboring intelligent agents to obtain the transformed intelligent agents, and determining the pursuit and escape micro-graph based on the transformed intelligent agents; determining the capture basin and the survival core according to the pursuit and escape micro-graph, initializing the graph neural network based on the capture basin and the survival core to obtain the initialized graph neural network; determining the weight domain of the initialized graph neural network based on the capture basin and the survival core, constraining the weight based on the weight domain, determining the target formation control information of the first intelligent agent, and performing formation control based on the target formation information to achieve qualitative and quantitative formation control.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of formation control technology, and in particular to a formation control method, device, equipment and storage medium for an intelligent agent network. Background Art

[0002] Formation control techniques have been widely used in various fields involving multi-agent network systems, such as satellite networks and autonomous ocean sampling networks. While consistency control algorithms aim to achieve consistency in the states of all agents in a dynamic network system, formation control algorithms go further by positioning agents based on relative states. Most existing multi-agent formation control methods can be divided into position-based, distance-based, displacement-based, and direction-based control. For example, distance-based and direction-based formation control achieves collision avoidance and connectivity maintenance. However, this method is limited by the switching graph topology and cannot achieve accurate formation control; the position-based and direction-based formation control problem uses navigation functions for collision avoidance and singularity avoidance, but this method is only applicable to low-speed agents and requires accurate sensors; the circular displacement-based unicycle agent formation control strategy uses the Lyapunov method for flow field analysis. This method has strict assumptions on the sensing range, making it impractical in real deployment. To date, there are few studies on optimal formation control for constrained nonlinear switched network systems, and convergence, collision avoidance, and connectivity maintenance are ensured when only local information is used. From this we can see that the current multi-agent formation control method is unable to perform qualitative and quantitative formation control.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a formation control method, device, equipment and storage medium for an intelligent agent network, aiming to solve the technical problem that the existing technology cannot perform qualitative and quantitative control on multi-agent formations.

[0005] To achieve the above objectives, the present application proposes a formation control method for an intelligent agent network, the formation control method for the intelligent agent network comprising:

[0006] Determine first state information of a first agent and second state information of a second agent in the agent network respectively, and determine a neighbor agent of the first agent according to the second state information, wherein the neighbor agent is an agent in the second agent connected to the first agent;

[0007] Generate an index set according to the first state information and the neighbor state information of the neighbor agent, and obtain a formation state based on the index set;

[0008] The agents corresponding to the formation state are uniformly transformed according to the local coordinate system of the neighboring agents to obtain transformed agents, and the chasing and escaping micro-graphs are determined based on the transformed agents;

[0009] Determine a capture basin and a survival core according to the pursuit and escape micrograph, and initialize the graph neural network based on the capture basin and the survival core to obtain an initialized graph neural network;

[0010] The weight domain of the initialized graph neural network is determined based on the capture basin and the survival kernel, the weights are constrained based on the weight domain, the target formation control information of the first agent is determined, and formation control is performed based on the target formation information.

[0011] In one embodiment, the step of generating an index set according to the first state information and the neighbor state information of the neighbor agent, and obtaining the formation state based on the index set includes:

[0012] Determine position information and direction information in first state information of a first agent, and determine a state vector of the first agent according to the position information and the direction information;

[0013] Generate a local coordinate system with a first agent, and determine a relative state and expected local formation information according to first state information of the first agent and neighbor state information of neighbor agents;

[0014] A formation state is obtained based on the relative state and the expected local formation information.

[0015] In one embodiment, the steps of uniformly transforming the agents corresponding to the formation state according to the local coordinate system of the neighboring agents to obtain transformed agents, and determining the pursuit and escape micro-graphs based on the transformed agents include:

[0016] According to the local coordinate system of the neighboring agent, the position information of the first agent is converted into the local coordinate system of the neighboring agent to obtain the position information of the first agent in the local coordinate system of the neighboring agent, and based on the position information of the first agent in the local coordinate system of the neighboring agent, the agents corresponding to the formation state are uniformly transformed to obtain the transformed agent by the rotation matrix corresponding to the direction information of the first agent;

[0017] The pursuit and escape micrographs are determined according to the capture basin and survival core corresponding to the transformed intelligent agent.

[0018] In one embodiment, the step of determining the pursuit and escape micrographs according to the capture basin and survival core corresponding to the transformed agent comprises:

[0019] determining a capture basin based on the state of the transforming agent;

[0020] Determine an index scale of the index set, and when the index scale is a first scale, determine the surviving core to be a first surviving core, and when the index scale is a second scale, determine the surviving core to be a second surviving core;

[0021] The capture basin and the first survival core or the second survival core are respectively associated with a time function and a maximum exit time function to obtain a pursuit and escape micrograph.

[0022] In one embodiment, the step of determining a capture basin and a survival core according to the pursuit and escape micrograph, and initializing a graph neural network based on the capture basin and the survival core to obtain an initialized graph neural network includes:

[0023] The capture basins and survival cores were determined based on the pursuit and escape micrographs described;

[0024] Determine a bounding parameter domain through the capture basin and the survival core based on a back propagation strategy, and continuously iterate the capture basin and the survival core based on the bounding parameter domain, stop iterating when the capture basin and the survival core remain unchanged, and obtain a target capture basin and a target survival core;

[0025] The graph neural network is initialized based on the target capture basin and the target survival core to obtain an initialized graph neural network.

[0026] In one embodiment, the step of initializing the graph neural network based on the target capture basin and the target survival core to obtain the initialized graph neural network includes:

[0027] Determining a shared variable in a graph neural network according to the target capture basin and the target survival kernel;

[0028] The graph neural network is initialized according to the shared variables to obtain an initialized graph neural network.

[0029] In one embodiment, before the step of performing formation control based on the target formation information, the method further includes:

[0030] performing a value estimation on the target formation control information of the first agent to determine an estimated value of the target formation control information;

[0031] Comparing the estimated value with an ideal value, and when the estimated value reaches the ideal value, executing the step of performing formation control based on the target formation information;

[0032] When the estimated value does not reach the ideal value, return to the step of constraining the weight based on the weight domain.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a formation control device for an intelligent agent network, the formation control device for the intelligent agent network comprising:

[0034] An agent determination module, used to respectively determine first state information of a first agent and second state information of a second agent in the agent network, and determine a neighboring agent of the first agent according to the second state information;

[0035] a formation determination module, configured to generate an index set according to the first state information and the neighbor state information of the neighbor agent, and obtain a formation state based on the index set, wherein the neighbor agent is an agent in the second agent connected to the first agent;

[0036] An agent transformation module, used to uniformly transform the agents corresponding to the formation state according to the local coordinate system of the neighboring agents to obtain transformed agents, and determine the pursuit and escape micro-graphs based on the transformed agents;

[0037] A network initialization module, used to determine a capture basin and a survival core according to the pursuit and escape micrograph, and initialize the graph neural network based on the capture basin and the survival core to obtain an initialized graph neural network;

[0038] A formation control module is used to determine the weight domain of the initialized graph neural network based on the capture basin and the survival kernel, constrain the weights based on the weight domain, determine the target formation control information of the first intelligent agent, and perform formation control based on the target formation information.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a formation control device for an intelligent agent network, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the formation control method for the intelligent agent network as described above.

[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the formation control method of the intelligent agent network as described above are implemented.

[0041] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the formation control method of the intelligent agent network as described above.

[0042] One or more technical solutions proposed in the present application have at least the following technical effects: respectively determining the first state information of the first agent and the second state information of the second agent in the agent network, determining the neighbor agent of the first agent according to the second state information, generating an index set according to the first state information and the neighbor state information of the neighbor agent, and obtaining a formation state based on the index set, wherein the neighbor agent is an agent in the second agent connected to the first agent, uniformly transforming the agents corresponding to the formation state according to the local coordinate system of the neighbor agent to obtain a transformed agent, and determining a pursuit and escape micrograph based on the transformed agent, determining a capture basin and a survival core according to the pursuit and escape micrograph, initializing a graph neural network based on the capture basin and the survival core to obtain an initialized graph neural network, determining a weight domain of the initialized graph neural network based on the capture basin and the survival core, constraining weights based on the weight domain, determining target formation control information of the first agent, and performing formation control based on the target formation information, thereby realizing qualitative and quantitative formation control of the agents in the agent network. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0045] Figure 1 A flow chart of the first embodiment of the formation control method of the intelligent agent network of the present application;

[0046] Figure 2 A flow chart of an embodiment of a formation control method for an intelligent agent network of the present application;

[0047] Figure 3 A schematic diagram of the module structure of a formation control device of an intelligent agent network according to an embodiment of the present application;

[0048] Figure 4 Schematic diagram of the device structure of the hardware operating environment involved in the formation control method of the intelligent agent network in the embodiment of the present application.

[0049] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0051] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0052] The main solution of the embodiment of the present application is: respectively determine the first state information of the first agent and the second state information of the second agent in the agent network, determine the neighbor agent of the first agent according to the second state information, wherein the neighbor agent is the agent in the second agent connected to the first agent, generate an index set according to the first state information and the neighbor state information of the neighbor agent, obtain the formation state based on the index set, uniformly transform the agents corresponding to the formation state according to the local coordinate system of the neighbor agent to obtain the transformed agent, and determine the pursuit and escape micro-graph based on the transformed agent, determine the capture basin and the survival core according to the pursuit and escape micro-graph, initialize the graph neural network based on the capture basin and the survival core to obtain the initialized graph neural network, determine the weight domain of the initialized graph neural network based on the capture basin and the survival core, constrain the weight based on the weight domain, determine the target formation control information of the first agent, and perform formation control based on the target formation information.

[0053] In this embodiment, for ease of description, the following description is made by taking the formation control device of the identification agent network as the execution subject.

[0054] Since most of the multi-agent formation control methods in the existing technology can be divided into position-based, distance-based, displacement-based and direction-based control. For example, the formation control based on distance and direction achieves collision avoidance and connectivity maintenance. However, this method is limited by the switching graph topology and cannot achieve precise formation control; for the formation control problem based on position and direction, a navigation function is used for collision avoidance and singularity avoidance, but this method is only applicable to low-speed agents and requires precise sensors; the unicycle agent formation control strategy based on circular displacement uses the Lyapunov method for flow field analysis. This method has strict assumptions on the sensing range, making it impractical in real deployment. To date, there have been few studies on optimal formation control for constrained nonlinear switched network systems, and convergence, collision avoidance and connectivity maintenance are ensured using only local information.

[0055] The present application provides a solution, by respectively determining the first state information of the first agent and the second state information of the second agent in the agent network, determining the neighbor agent of the first agent according to the second state information, generating an index set according to the first state information and the neighbor state information of the neighbor agent, and obtaining a formation state based on the index set, wherein the neighbor agent is an agent in the second agent connected to the first agent, performing a uniform transformation on the agents corresponding to the formation state according to the local coordinate system of the neighbor agent to obtain a transformed agent, and determining a pursuit and escape micrograph based on the transformed agent, determining a capture basin and a survival core according to the pursuit and escape micrograph, initializing a graph neural network based on the capture basin and the survival core to obtain an initialized graph neural network, determining a weight domain of the initialized graph neural network based on the capture basin and the survival core, constraining the weight based on the weight domain, determining the target formation control information of the first agent, and performing formation control based on the target formation information, thereby realizing qualitative and quantitative formation control of the agents in the agent network.

[0056] It can be seen from the above embodiments that the present application determines the first state information of the first agent and the second state information of the second agent in the agent network respectively, determines the neighbor agent of the first agent according to the second state information, generates an index set according to the first state information and the neighbor state information of the neighbor agent, obtains the formation state based on the index set, wherein the neighbor agent is the agent in the second agent connected to the first agent, uniformly transforms the agents corresponding to the formation state according to the local coordinate system of the neighbor agent to obtain the transformed agent, determines the pursuit and escape micro-graph based on the transformed agent, determines the capture basin and the survival core according to the pursuit and escape micro-graph, initializes the graph neural network based on the capture basin and the survival core to obtain the initialized graph neural network, determines the weight domain of the initialized graph neural network based on the capture basin and the survival core, constrains the weight based on the weight domain, determines the target formation control information of the first agent, and performs formation control based on the target formation information, thereby realizing qualitative and quantitative formation control of the agents in the agent network.

[0057] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a formation control device of an intelligent network, etc. The following takes the formation control device of an intelligent network as an example to illustrate this embodiment and the following embodiments.

[0058] Based on this, the embodiment of the present application provides a formation control method for an intelligent agent network, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the formation control method of the intelligent agent network of the present application.

[0059] In this embodiment, the formation control method of the agent network includes steps S10 to S50:

[0060] Step S10, respectively determine the first state information of the first agent and the second state information of the second agent in the agent network, and determine the neighbor agent of the first agent according to the second state information, wherein the neighbor agent is an agent in the second agent connected to the first agent.

[0061] It should be noted that the agent network is a network formed by multiple independent agents that perceive the environment and connect to each other through the network based on the perceived environmental information to exchange information to solve a problem or achieve a goal. The agent network has the characteristics of distribution, autonomy, interactivity, adaptability and scalability. In the agent network, the currently determined agent can be defined as the first agent, and the agents other than the first agent in other agent networks are defined as the second agent, and the first state information and the second state information are the current state information of the first agent and the second agent, and the state information at least includes the location information and direction information of the first agent.

[0062] It can be understood that in the intelligent agent network, no intelligent agent is isolated, so for any intelligent agent, there is data interaction with at least one second intelligent agent. Therefore, the second intelligent agents that have data interaction with the first intelligent agent are defined as neighboring intelligent agents, and the neighbor status information reflects the status information of the current neighboring intelligent agents.

[0063] In a specific implementation, when determining an agent in an agent network, the current agent network can be traversed to determine the information of each agent in the current agent network, and the topological relationship between agents can be generated as needed. The state information of the agent can be preloaded during the traversal process based on the determined topological relationship. When determining the first agent and the second agent in the agent network, an agent can be arbitrarily selected as the first agent, and the state information corresponding to the agent can be determined as the first state information. All agents in the agent network except the current first agent are defined as second agents, and the state information corresponding to the second agent is determined as the second state information. Then, based on the topological relationship between agents, the second agent that has data interaction with the first agent is determined as the neighbor agent.

[0064] Step S20: Generate an index set according to the first state information and the neighbor state information of the neighbor agent, and obtain a formation state based on the index set.

[0065] It should be noted that the index set refers to the set of index relationships between the first agent and its neighboring agents. The formation state refers to a collective structure formed between agents.

[0066] In a specific implementation, by determining the agents in the agent network, determining the connection relationship between the first agent and its neighboring agents, and determining the index information through the data interaction relationship between the first agent and the neighboring agents, the obtained index information is summarized to obtain an index set, wherein the index information is vector information for indicating the direction of data interaction. For the agent, the expected relative state of the first agent in the local formation can be determined based on the state information of the neighboring agents and the first agent, and by confirming the communication area, collision avoidance and target set of the agent, it can be determined whether the first agent can effectively communicate when realizing formation control, avoid collisions and ultimately achieve the expected formation goal.

[0067] Step S30, uniformly transforming the agents corresponding to the formation state according to the local coordinate system of the neighboring agents to obtain transformed agents, and determining the pursuit and escape micro-graphs based on the transformed agents.

[0068] It should be noted that in a multi-agent system, each agent has a local coordinate system, which is usually centered on the agent itself. The agent can perceive the position of its neighbors in its own local coordinate system. The chase and escape micrograph describes a strategy or model based on the relative positions and behaviors between agents. In a chase and escape scenario, some agents may play the role of pursuers, while others play the role of escapers. The chase and escape micrograph is used to describe this chase and escape relationship between agents.

[0069] In the specific implementation, when the agents corresponding to the formation state are uniformly transformed according to the local coordinate system of the neighboring agents, an agent is transformed from its own local coordinate system to a unified coordinate system. This transformation usually involves translation (changing position) and rotation (changing direction) operations, so that the position and direction of the agent in the formation match the overall form of the formation. After the coordinate transformation, the position and direction of the agent in the unified coordinate system are called the transformed agent. This transformed agent represents the relative position and direction of the agent in the formation relative to other agents. After the unified transformation is completed, their strategies in the chase and escape game are determined according to their positions and directions in the unified coordinate system. For example, the chaser agent may adjust its pursuit path according to the position of the escaper agent, while the escaper agent may plan the escape route according to the position of the chaser, thereby generating a chase and escape micro-map.

[0070] Step S40, determining a capture basin and a survival core according to the pursuit and escape micrograph, and initializing the graph neural network based on the capture basin and the survival core to obtain an initialized graph neural network.

[0071] It should be noted that the capture basin refers to the area or conditions where the pursuer can successfully capture the evader in the chase and escape scenario. This is usually related to factors such as the agent's perception range, movement speed, and strategy. The survival core is opposite to the capture basin and refers to the area or conditions where the evader can continue to evade the pursuer. This may involve the evader's hiding ability, speed advantage, or familiarity with the environment.

[0072] In the specific implementation, graph neural networks are mainly used to solve qualitative and quantitative problems of chase and escape differential games. Qualitative problems include estimating the capture basin and survival kernel, which determine the parameter domain of the approximate value function. In quantitative problems, approximate estimation is included, that is, estimating the weight of GNN, and then designing the optimal control strategy of differential graph games. GNN refers to graph neural network, which is a deep learning model that specializes in processing graph structure data. The graph neural network is initialized based on the capture basin and survival kernel, and adjusted through reverse set propagation according to the Pontryagin minimum principle and maximum principle in the back-propagation process to obtain the initialized graph neural network.

[0073] Step S50, determining the weight domain of the initialized graph neural network based on the capture basin and the survival kernel, constraining the weights based on the weight domain, determining the target formation control information of the first intelligent agent, and performing formation control based on the target formation information.

[0074] In the specific implementation, the parameters of the estimated value function can be initialized in an offline manner, and then online learning is performed to achieve real-time optimization in the form of a multi-agent pursuit and escape differential graph game to complete the formation control task. The approximate values ​​of the capture basin and the survival kernel are used to initialize the graph neural network, and the weight domain of the initialized graph neural network is defined. The weight domain of the shared weights in the entire network is adjusted in an online manner through a collaborative learning algorithm, and the formation control is performed based on the target formation control information of the first agent.

[0075] The present embodiment provides a formation control method for an agent network, by respectively determining first state information of a first agent and second state information of a second agent in the agent network, determining a neighbor agent of the first agent according to the second state information, generating an index set according to the first state information and the neighbor state information of the neighbor agent, and obtaining a formation state based on the index set, wherein the neighbor agent is an agent in the second agent connected to the first agent, performing a uniform transformation on the agents corresponding to the formation state according to the local coordinate system of the neighbor agent to obtain a transformed agent, and determining a pursuit and escape micrograph based on the transformed agent, determining a capture basin and a survival core according to the pursuit and escape micrograph, initializing a graph neural network based on the capture basin and the survival core to obtain an initialized graph neural network, determining a weight domain of the initialized graph neural network based on the capture basin and the survival core, constraining weights based on the weight domain, determining target formation control information of the first agent, and performing formation control based on the target formation information, thereby achieving qualitative and quantitative formation control of the agents in the agent network.

[0076] In one possible embodiment, referring to Figure 2 , step S20 may include steps S21 to S23:

[0077] S21: Determine position information and direction information in first state information of a first agent, and determine a state vector of the first agent according to the position information and the direction information.

[0078] S22: Generate a local coordinate system with the first agent, and determine the relative state and expected local formation information according to the first state information of the first agent and the neighbor state information of the neighboring agents.

[0079] S23: Obtaining a formation state based on the relative state and the expected local formation information.

[0080] In the specific implementation, it is possible to set the intelligent agent to include N intelligent agents. The agent is represented as .and The index set of connected neighbor agents is represented as In the global coordinate system, Agent The state is represented by , including its location and direction (The present invention only considers two-dimensional space, and three-dimensional space can also be processed similarly.) The intelligent agent considered in the present invention is an autonomous vehicle with the following dynamics:

[0081] .

[0082] in, represents the state vector of the agent, represents the control input. The dynamics of all agents in the network are controllable and identical; the function and is uniformly continuous and differentiable, which is applicable to almost all autonomous vehicle systems.

[0083] The network system can be divided into pairs of connected agents. Since only local information can be used, Expressed as a clockwise rotation matrix, the agent and The relative status between Can be In the local coordinate system of ,in .

[0084] The desired local formation frame is given by Desired relative state in local coordinate system Build, where and In the global formation framework, and Then, is the relative state of the local formation. These pairs of agents have the following properties:

[0085] Communication Region: Pairs of Agents and Only in communication area Communicate within .

[0086] Collision avoidance: When considering the security of the network system, only collision avoidance between agents is considered. Expressed as If the collision set ,but and There was a collision between them.

[0087] Target set: Expressed as The target set contains the origin and is based on The design is performed with the allowable convergence deviation. If for all , ,have , then the network formation is achieved.

[0088] These properties ensure that the agents can communicate effectively, avoid collisions, and ultimately achieve the desired formation goals when implementing formation control.

[0089] By determining the topological relationship between intelligent agents, an accurate description relationship is provided for formation control. The communication area, collision avoidance, and target set between intelligent agents are integrated to determine the formation state of the intelligent agents. The formation state can be the expected formation state, thereby improving the quality of the formation.

[0090] In a feasible embodiment, the steps of uniformly transforming the agents corresponding to the formation state according to the local coordinate system of the neighboring agents to obtain transformed agents, and determining the pursuit and escape micro-graphs based on the transformed agents include:

[0091] According to the local coordinate system of the neighboring agent, the position information of the first agent is converted into the local coordinate system of the neighboring agent, and the position information of the first agent in the local coordinate system of the neighboring agent is obtained, and based on the position information of the first agent in the local coordinate system of the neighboring agent, the agents corresponding to the formation state are uniformly transformed, and the rotation matrix corresponding to the direction information of the first agent is obtained to obtain the transformed agent;

[0092] The pursuit and escape micrographs are determined according to the capture basin and survival core corresponding to the transformed intelligent agent.

[0093] In the specific implementation, since only the states of the paired agents are considered in the formula (i.e. and ), so the system dynamics of all connected agents is transformed into the system dynamics of pairs of agents. and ) can be rewritten as:

[0094] .

[0095] When the intelligent The local coordinate system is transformed into When the local coordinate system is . All connected agent pairs ( and ) can be unified by the following transformation: hour ,when hour ,in, is a clockwise rotation matrix. After the local coordinate systems of the agent pair are unified, the system can be transformed into:

[0096] .

[0097] Among them, when hour, , = ,when hour, , = In this way, by converting the system dynamics into the relative state dynamics of pairs of intelligent agents, the control and analysis of the entire network system can be simplified, making it easier to design control strategies to ensure the realization of network formation goals. Constructing the pursuit and escape micro-graph game strategy. This means , . Therefore, the loss function in the differential graph game is defined as:

[0098] .

[0099] The step of determining the pursuit and escape micrographs according to the capture basin and the survival core corresponding to the transformed intelligent agent comprises:

[0100] determining a capture basin based on the state of the transforming agent;

[0101] Determine an index scale of the index set, and when the index scale is a first scale, determine the surviving core to be a first surviving core, and when the index scale is a second scale, determine the surviving core to be a second surviving core;

[0102] The capture basin and the first survival core are respectively associated with the time function and the maximum exit time function, or the capture basin and the second survival core are respectively associated with the time function and the maximum exit time function to obtain a pursuit and escape micrograph.

[0103] In the specific implementation, in determining the capture basin according to the state of the transformed agent, the connectivity of the graph and the safety of the network are determined by the capture basin and the survival kernel respectively, because they characterize the controllability of the network system. The definitions are as follows:

[0104]

[0105] Survival Core The definition is as follows: hour:

[0106] .

[0107] Determine the index scale of the index set, when the index scale of the index set is the first scale, that is, when hour:

[0108]

[0109] Using the capture basin and survival kernel, we can correlate the arrival time function

[0110]

[0111] And the maximum exit time function: when hour:

[0112]

[0113] When the index scale of the index set is the first scale, that is, when hour:

[0114]

[0115] This embodiment uniformly transforms the first agent based on the local coordinate system of the neighboring agents, determines the capture basin and survival core of the transformed agents, and formulates the optimal formation control problem as a multi-player pursuit and escape differential graph game based on survival theory. By utilizing local information, it ensures that all agents can reach the required formation at the fastest convergence rate within a limited time and meet all constraints.

[0116] In a feasible embodiment, the step of determining a capture basin and a survival core according to the pursuit and escape micrograph, initializing the graph neural network based on the capture basin and the survival core, and obtaining the initialized graph neural network includes:

[0117] The capture basins and survival cores were determined based on the pursuit and escape micrographs described;

[0118] By back propagation, a bounding parameter domain is determined according to the capture basin and the survival core, and the capture basin and the survival core are continuously iterated based on the bounding parameter domain. When the capture basin and the survival core remain unchanged, the iteration is stopped to obtain a target capture basin and a target survival core;

[0119] The graph neural network is initialized based on the target capture basin and the target survival core to obtain an initialized graph neural network.

[0120] The step of initializing the graph neural network based on the target capture basin and the target survival core to obtain the initialized graph neural network includes:

[0121] Determining a shared variable in a graph neural network according to the target capture basin and the target survival kernel;

[0122] The graph neural network is initialized according to the shared variables to obtain an initialized graph neural network.

[0123] In the specific implementation, graph neural networks are mainly used to solve qualitative and quantitative problems of chase and escape differential games. Qualitative problems include estimating capture basins and survival core , which determine the approximate value function The parameter domain of , that is, to estimate the weight of GNN, and then design the optimal control strategy for differential graph game. The architecture of graph neural network is designed as follows:

[0124]

[0125] in , and Expressed as an approximate survival kernel, if ,but , otherwise if ,but . Similarly, if and only if hour, ,in is an approximate capture basin. , and is a shared variable in the network.

[0126] The goals of parameter initialization are twofold:

[0127] Define the parameter domain within which a feasible control strategy exists, i.e., capture basin and survival core .

[0128] Initialize the parameters so that the initial control strategy can ensure the connectivity, convergence and security of the network system.

[0129] Through reverse set propagation, The parameters are initialized in the approximate set and In which and To approximate and , the data set propagates from and Start. Spread and When , the parameters are adjusted according to the Pontryagin minimum principle and maximum principle. It can be defined in the following ways:

[0130]

[0131] The survival core can be defined in the following ways:

[0132]

[0133] The propagation of the capture basin and survival kernel approximation is terminated when the boundaries of the backpropagation dataset remain unchanged after consecutive iterations. The result of the backpropagation of the dataset is , Once the boundary of the propagation set passes through a set of states, then Will be initialized by the following HJB:

[0134]

[0135] parameter Will be initialized by the following HJB:

[0136]

[0137] parameter Initialized to 1.

[0138] In a feasible embodiment, before the step of performing formation control based on the target formation information, the method further includes:

[0139] performing a value estimation on the target formation control information of the first agent to determine an estimated value of the target formation control information;

[0140] Comparing the estimated value with an ideal value, and when the estimated value reaches the ideal value, executing a step of performing formation control based on the target formation information;

[0141] When the estimated value does not reach the ideal value, return to the step of constraining the weight based on the weight domain.

[0142] In the specific implementation, by estimating the value function The parameters of are initialized in an offline manner, and then online learning is performed to achieve real-time optimization in the form of a multi-agent pursuit and escape differential graph game to complete the formation control task. The approximate values ​​of the capture basin and the survival kernel are used to initialize the neural network and , and define and domain (i.e. and ). The collaborative learning algorithm adjusts the weights shared across the entire network in an online manner. and In control strategy design:

[0143] ;

[0144] ;

[0145] when When the control strategy is designed to ensure The convergence of When , the designed control strategy aims to ensure collision avoidance and connectivity, i.e. However, when two independent control strategies are used, optimal convergence cannot be achieved. In this sense, the optimal control strategy can only rely on the first control strategy without using the second one. pass When adaptive, ideally , so only the first control strategy needs to be adopted. and The proposed adaptive algorithms ensure that they can always converge to the ideal value with a bounded convergence error:

[0146]

[0147]

[0148] in The forgetting factor is adjusted between 0 and 1. and It is continuously motivating. and Initialize to positive proof, and The calculations are as follows:

[0149] ,

[0150] ,

[0151]

[0152] In this embodiment, the value function can be estimated through GNN online learning to achieve real-time optimization of the completed formation control task, and the optimal formation control can be achieved while the intelligent agent maintains network connectivity, convergence and security according to the control strategy.

[0153] When calculating the estimated value function, its Bellman error can be determined. The specific process can be described as:

[0154] Since the value function is fitted by the graph neural network:

[0155]

[0156] in:

[0157]

[0158] and is the structural error of the graph neural network. After verification, the value function is obtained is the viscosity solution of the following quasi-variational Hamilton-Jacobi-Bellman (HJB) inequality

[0159]

[0160]

[0161] The calculation of Bellman error is as follows:

[0162]

[0163] in,

[0164]

[0165] when Approaching When , the estimation error reaches the ideal value, where

[0166]

[0167] By calculating the Bellman error and evaluating and improving the approximate accuracy of the value function, it is possible to ensure that the value function satisfies the HJB inequality, providing theoretical support and error bounds for the optimal control strategy.

[0168] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the formation control method of the intelligent agent network of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0169] This application also provides a formation control device for an intelligent agent network, please refer to Figure 3 , the formation control device of the agent network includes:

[0170] An agent determination module 10 is used to respectively determine first state information of a first agent and second state information of a second agent in the agent network, and determine a neighbor agent of the first agent according to the second state information, wherein the neighbor agent is an agent in the second agent connected to the first agent;

[0171] A formation determination module 20, configured to generate an index set according to the first state information and the neighbor state information of the neighbor agent, and obtain a formation state based on the index set;

[0172] An agent transformation module 30 is used to uniformly transform the agents corresponding to the formation state according to the local coordinate system of the neighboring agents to obtain transformed agents, and determine the pursuit and escape micro-graphs based on the transformed agents;

[0173] A network initialization module 40 is used to determine a capture basin and a survival core according to the pursuit and escape micrograph, and initialize the graph neural network based on the capture basin and the survival core to obtain an initialized graph neural network;

[0174] The formation control module 50 is used to determine the weight domain of the initialized graph neural network based on the capture basin and the survival kernel, constrain the weights based on the weight domain, determine the target formation control information of the first intelligent agent, and perform formation control based on the target formation information.

[0175] The formation control device of the intelligent agent network provided by the present application adopts the formation control method of the intelligent agent network in the above embodiment, which can solve the technical problem that the prior art cannot perform qualitative and quantitative control on the multi-agent formation. Compared with the prior art, the beneficial effects of the formation control device of the intelligent agent network provided by the present application are the same as the beneficial effects of the formation control method of the intelligent agent network provided by the above embodiment, and the other technical features in the formation control device of the intelligent agent network are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0176] The present application provides a formation control device for an intelligent agent network, the formation control device for the intelligent agent network comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the formation control method for the intelligent agent network in the above-mentioned embodiment one.

[0177] Reference below Figure 4, which shows a schematic diagram of the structure of a formation control device of an intelligent agent network suitable for implementing an embodiment of the present application. The formation control device of the intelligent agent network in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The formation control device of the intelligent agent network shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0178] like Figure 4 As shown, the formation control device of the agent network may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the formation control device of the agent network are also stored in RAM1004. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the formation control device of the intelligent agent network to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a formation control device of an intelligent agent network with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0179] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0180] The formation control device of the intelligent agent network provided by the present application adopts the formation control method of the intelligent agent network in the above embodiment, which can solve the technical problem that the prior art cannot perform qualitative and quantitative control on the multi-agent formation. Compared with the prior art, the beneficial effects of the formation control device of the intelligent agent network provided by the present application are the same as the beneficial effects of the formation control method of the intelligent agent network provided by the above embodiment, and the other technical features in the formation control device of the intelligent agent network are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0181] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0182] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0183] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, the computer-readable program instructions being used to execute the formation control method for an intelligent agent network in the above-mentioned embodiment.

[0184] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0185] The computer-readable storage medium may be included in the formation control device of the intelligent agent network; or may exist independently without being assembled into the formation control device of the intelligent agent network.

[0186] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a formation control device of an agent network, the formation control device of the agent network:

[0187] Determine first state information of a first agent and second state information of a second agent in the agent network respectively, and determine a neighbor agent of the first agent according to the second state information, wherein the neighbor agent is an agent in the second agent connected to the first agent;

[0188] a formation determination module, configured to generate an index set according to the first state information and the neighbor state information of the neighboring agents, and obtain a formation state based on the index set;

[0189] The agents corresponding to the formation state are uniformly transformed according to the local coordinate system of the neighboring agents to obtain transformed agents, and the chasing and escaping micro-graphs are determined based on the transformed agents;

[0190] Determine a capture basin and a survival core according to the pursuit and escape micrograph, and initialize the graph neural network based on the capture basin and the survival core to obtain an initialized graph neural network;

[0191] The weight domain of the initialized graph neural network is determined based on the capture basin and the survival kernel, the weights are constrained based on the weight domain, the target formation control information of the first agent is determined, and formation control is performed based on the target formation information.

[0192] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0193] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0194] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0195] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned formation control method of the agent network, and can solve the technical problem that the prior art cannot perform qualitative and quantitative control on the multi-agent formation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the formation control method of the agent network provided by the above-mentioned embodiment, and will not be repeated here.

[0196] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the formation control method of the intelligent agent network as described above.

[0197] The computer program product provided by the present application can solve the technical problem that the prior art cannot perform qualitative and quantitative control on a multi-agent formation. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the formation control method of the agent network provided by the above embodiment, and will not be repeated here.

[0198] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A formation control method for an intelligent agent network, characterized in that: The formation control method of the agent network includes: Determine first state information of a first agent and second state information of a second agent in the agent network respectively, and determine a neighbor agent of the first agent according to the second state information, wherein the neighbor agent is an agent in the second agent connected to the first agent; Generate an index set according to the first state information and the neighbor state information of the neighbor agent, and obtain a formation state based on the index set; The agents corresponding to the formation state are uniformly transformed according to the local coordinate system of the neighboring agents to obtain transformed agents, and the chasing and escaping micro-graphs are determined based on the transformed agents; Determine a capture basin and a survival core according to the pursuit and escape micrograph, and initialize the graph neural network based on the capture basin and the survival core to obtain an initialized graph neural network; The weight domain of the initialized graph neural network is determined based on the capture basin and the survival kernel, the weights are constrained based on the weight domain, the target formation control information of the first agent is determined, and formation control is performed based on the target formation control information.

2. The method according to claim 1, characterized in that The step of generating an index set according to the first state information and the neighbor state information of the neighboring intelligent agent, and obtaining the formation state based on the index set comprises: Determine position information and direction information in first state information of a first agent, and determine a state vector of the first agent according to the position information and the direction information; Generate a local coordinate system with a first agent, and determine a relative state and expected local formation information according to first state information of the first agent and neighbor state information of neighbor agents; A formation state is obtained based on the relative state and the expected local formation information.

3. The method according to claim 1, characterized in that The steps of uniformly transforming the agents corresponding to the formation state according to the local coordinate system of the neighboring agents to obtain the transformed agents, and determining the chasing and escaping micro-graphs based on the transformed agents include: According to the local coordinate system of the neighboring agent, the position information of the first agent is converted into the local coordinate system of the neighboring agent to obtain the position information of the first agent in the local coordinate system of the neighboring agent, and based on the position information of the first agent in the local coordinate system of the neighboring agent, the agents corresponding to the formation state are uniformly transformed to obtain the transformed agent by the rotation matrix corresponding to the direction information of the first agent; The pursuit and escape micrographs are determined according to the capture basin and survival core corresponding to the transformed intelligent agent.

4. The method according to claim 3, characterized in that The step of determining the pursuit and escape micrographs according to the capture basin and survival core corresponding to the transformed intelligent agent comprises: determining a capture basin based on the state of the transforming agent; Determine an index scale of the index set, and when the index scale is a first scale, determine the surviving core to be a first surviving core, and when the index scale is a second scale, determine the surviving core to be a second surviving core; The capture basin and the first survival core or the second survival core are respectively associated with a time function and a maximum exit time function to obtain a pursuit and escape micrograph.

5. The method according to claim 1, characterized in that The steps of determining a capture basin and a survival core according to the pursuit and escape micrograph, initializing the graph neural network based on the capture basin and the survival core, and obtaining the initialized graph neural network include: The capture basins and survival cores were determined based on the pursuit and escape micrographs described; Determine a bounding parameter domain through the capture basin and the survival core based on a back propagation strategy, and continuously iterate the capture basin and the survival core based on the bounding parameter domain, stop iterating when the capture basin and the survival core remain unchanged, and obtain a target capture basin and a target survival core; The graph neural network is initialized based on the target capture basin and the target survival core to obtain an initialized graph neural network.

6. The method according to claim 5, characterized in that The step of initializing the graph neural network based on the target capture basin and the target survival core to obtain the initialized graph neural network includes: Determine a shared variable in a graph neural network according to the target capture basin and the target survival kernel; The graph neural network is initialized according to the shared variables to obtain an initialized graph neural network.

7. The method according to claim 1, characterized in that Before the step of performing formation control based on the target formation control information, the method further includes: estimating the value of the target formation control information of the first agent to determine an estimated value of the target formation control information; Comparing the estimated value with an ideal value, and when the estimated value reaches the ideal value, executing the step of performing formation control based on the target formation control information; When the estimated value does not reach the ideal value, return to the step of constraining the weight based on the weight domain.

8. A formation control device for an intelligent agent network, characterized in that: The device comprises: An agent determination module, used to respectively determine first state information of a first agent and second state information of a second agent in the agent network, and determine a neighboring agent of the first agent according to the second state information; a formation determination module, configured to generate an index set according to the first state information and the neighbor state information of the neighbor agent, and obtain a formation state based on the index set, wherein the neighbor agent is an agent in the second agent connected to the first agent; An agent transformation module, used to uniformly transform the agents corresponding to the formation state according to the local coordinate system of the neighboring agents to obtain transformed agents, and determine the pursuit and escape micro-graphs based on the transformed agents; A network initialization module, used to determine a capture basin and a survival core according to the pursuit and escape micrograph, and initialize the graph neural network based on the capture basin and the survival core to obtain an initialized graph neural network; A formation control module is used to determine the weight domain of the initialized graph neural network based on the capture basin and the survival kernel, constrain the weights based on the weight domain, determine the target formation control information of the first intelligent agent, and perform formation control based on the target formation control information.

9. A formation control device for an intelligent agent network, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the formation control method for an intelligent agent network as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the formation control method of the intelligent agent network as described in any one of claims 1 to 7 are implemented.