Multi-agent system preset time formation control method based on fuzzy reinforcement learning

Abstract

The invention discloses a multi-agent system preset time formation control method based on fuzzy reinforcement learning. The method comprises the following steps: establishing a communication network among multiple agents based on a graph theory; constructing a fuzzy logic system; constructing a multi-agent system; an auxiliary function and an error transformation function are constructed with the aim that multiple agents complete target formation control within the preset time; defining an optimal formation control theory, determining an optimal performance index function, and introducing an HJB equation to solve optimal control input; and applying an ACI structure, designing an identifier, a criticer, an actor and a corresponding updating rule, and completing parameter matrix updating of optimal formation control. According to the invention, the robust control of the multi-agent system can be realized in a complex environment, so that the multi-agent system can reach the target formation within the preset time, and the anti-interference capability is high.

Description

Technical Field

[0001] The present application relates to the technical field of multi-agent system formation control, and specifically to a multi-agent system preset time formation control method based on fuzzy reinforcement learning. Background Art

[0002] With the continuous advancement of science and technology, multi-agent systems are increasingly used in areas such as drone formations and robot teams. However, in these complex dynamic environments, achieving efficient coordination and formation control between agents still faces many challenges. Traditional multi-agent system control methods often perform poorly when dealing with dynamic uncertainty and external interference, making it difficult to ensure that the system achieves the expected formation effect within the specified time. In addition, environmental uncertainty and nonlinear interactions between agents further increase the difficulty of control.

[0003] To solve these problems, there is a design of a robust preset time controller combining a neural network with an output regulator in the prior art to implement a preset time tracking control method for a heterogeneous multi-agent system based on a neural network. Although it can optimize the collaborative strategy of the agents through an adaptive learning mechanism to ensure that the system reaches the target formation within the preset time in a complex environment, such methods still have limitations in dealing with uncertainties such as system parameters or environmental changes. There is also a control strategy for multi-agents by designing a fuzzy logic system in the prior art. According to the ability of the fuzzy logic system to handle uncertainty and fuzzy information, a dynamic adaptive update rate is provided to improve the efficiency of the multi-agent formation, but such methods have limitations in terms of collaborative control between agents.

[0004] In response to the above problems, the present invention proposes a robust control method for preset time formation of a multi-agent system based on fuzzy reinforcement learning, which effectively overcomes the limitations of traditional control methods, improves the adaptability and control accuracy of the multi-agent system in a dynamic environment, and provides strong technical support for the formation tasks of multi-agent systems such as drone swarms and robot teams. Summary of the invention

[0005] In order to achieve robust control of a multi-agent system in a complex environment, thereby ensuring that the system reaches the target formation within a preset time and has strong anti-interference ability, the present application provides a preset time formation control method for a multi-agent system based on fuzzy reinforcement learning.

[0006] In a first aspect, the present application provides a method for controlling a multi-agent system with a preset time formation based on fuzzy reinforcement learning, comprising: S1. Based on graph theory, establish a communication network between multiple agents; S2, build fuzzy logic system; S3. Constructing a multi-agent system, including: constructing a state equation of the multi-agent, constructing a formation error equation based on the multi-agent state equation, and designing a performance indicator function that introduces a formation error; S4, with the goal of multi-agents completing target formation control within a preset time, construct auxiliary functions and error transformation functions; S5. Define the optimal formation control theory, including defining the optimal performance index function after updating the performance index function using the error change function, and introduce the HJB equation to solve the optimal control input; S6. Apply the ACI structure and combine the fuzzy logic system to design identifiers, critics, actors and corresponding update rules to complete the parameter matrix update of the optimal formation control.

[0007] By adopting the above scheme, the fuzzy logic system is used to accurately calculate the output of multi-agents in a complex dynamic environment; auxiliary functions are constructed to assist the multi-agent system to complete the precise control of the formation within a preset time; the optimal formation control theory is combined to obtain the optimal control input; the update rules of identifiers, critics and actors are designed to realize the parameter matrix update of the optimal formation control and achieve robust control.

[0008] Preferably, the S1 step specifically includes: S11. Define vertex set , Indicates the number of agents; S12. Define edge set , Represents the communication connection between agents; S13. Define the adjacency matrix , Representing an Agent With Agent When there is a communication connection between ,otherwise, ; S14. Define an undirected connected graph , Characterize the communication network topology of a multi-agent system; where , represents the Laplacian matrix between the agent and the leader; represents the communication matrix between the agent and the leader, where it is assumed that at least one agent is connected to the leader, that is, .

[0009] By adopting the above scheme, the vertex set, edge set, adjacency matrix and undirected connected graph are defined to effectively characterize the communication network structure of the multi-agent system, ensuring that each agent can accurately receive and send information, avoiding communication delays or interruptions, and thus improving the formation control performance of the entire multi-agent system.

[0010] Preferably, the S2 step specifically includes: S21, establish a fuzzy rule base; the rule form in the fuzzy rule base is: yes , yes , yes ,but yes ;in, Indicates input, Indicates the output, represents the number of fuzzy rules; Representing fuzzy sets , express The membership function of S22, using single-state fuzzification, product reasoning and central average defuzzification to calculate output ; The calculation formula includes: in, is the total number of fuzzy rules; satisfy ,make in, and , It is expressed as: .

[0011] By adopting the above scheme, a detailed fuzzy rule base is established and the methods of single-state fuzzification, product reasoning and central average defuzzification are adopted to improve the computational efficiency and enhance the robustness and stability of the system.

[0012] Preferably, the S3 step comprises: S31. Construct the state equation of the multi-agent system. The formula is: in, Indicates the system status. Indicates the control output, represents an unknown continuous nonlinear function, Indicates external interference; S32. Construct coordinate transformation equation: in, is the tracking error, For the leader status, Represents leaders and agents The relative position of S33. Construct the formation error equation, the formula is: in, , if there is a protocol , so that for , is a stable time, and is the expected accuracy; represents the adjacency matrix of the ith follower agent; represents the communication weight between the ith agent and its neighboring agents; represents the communication connection weight between the leader and the i-th agent; S34. Construct a performance indicator function, the formula is: in, represents the discount factor, and represents a symmetric positive definite matrix; for a multi-agent system, if is continuous, , In the collection On stable, is finite, then Admission Control Protocol , expressed as .

[0013] By adopting the above scheme, the state of each intelligent agent and its dynamic characteristics are constructed in detail, and the relative position relationship between the leader and the intelligent agent is reflected through the coordinate transformation equation, thereby quantifying the error in the formation process. Combined with the constructed performance indicators, it provides a basis for the subsequent completion of robust control of the formation within the preset time.

[0014] Preferably, the S4 step comprises: S41. In order to realize the preset time control performance of the multi-agent system, construct an auxiliary function M: in, is the design parameter, is the specified time; M is is strictly decreasing, and when hour, ,and ; M is smooth, M is All times are bounded; S42. Constructing error transformation function , The construction is as follows: in, is a constant, and .

[0015] By adopting the above scheme, it is ensured that the constructed auxiliary function strictly decreases within the preset time and eventually tends to zero. The auxiliary function is used to adjust the error transformation function, so as to effectively control the formation process of the multi-agent system to reach the target formation state more accurately within the predetermined time.

[0016] Preferably, the step S5 comprises: S51, updating the performance index function according to the error transformation function, which is further expressed as follows after the update: in, pass Expressed as , and Substitute income; S52, optimal group control , and get the optimal performance index function: ; S53, calculate the time derivative of the optimal performance index function and obtain the HJB equation: in, , , ,and is the only solution to the HJB equation. When , the optimal control input is obtained: ; S54. In order to achieve optimal group control, Split into: in, is a design parameter, , ; in, , , ,Will Substitution , and obtain the optimal control input: ; Among them, the unknown parameters and is continuous, for and ,exist and So that: in, and All represent the optimal parameter matrix; and is the fuzzy rule number; and represents the fuzzy basis function vector; approximation error and Corresponding satisfaction and ; and is a constant.

[0017] By adopting the above scheme, the time derivative of the optimal performance index function is calculated, the HJB equation is derived, and the equation is solved to obtain the optimal control input, ensuring that the multi-agent system achieves optimal formation control within the preset time; the optimal control input is divided into multiple parts, and the influence of design parameters and fuzzy basis functions are considered separately to further ensure the accuracy and robustness of the control output.

[0018] Preferably, the S6 step comprises: S61. Apply the ACI structure and design the identifier as follows: in, and They represent the output of FLS and identifier parameter matrix respectively, and the design identifier update rule is: in, represents the design parameters, represents a positive definite matrix; S62. Design a critic to evaluate control performance, the formula is: in, is the critic parameter matrix; the critic update rule is designed as: in, and is the design parameter; S63. Design actors to achieve control behavior. The formula is: in, Represents the actor parameter matrix, and the actor update rule is designed as: in, is the design parameter; S64, if If the value of the feature vector falls on 0, the training is terminated.

[0019] By adopting the above scheme, the network update law of the control strategy is designed to realize the update of the control parameter matrix and ensure the robustness of the control input.

[0020] Preferably, the step S41 further includes: Calculate the output of each intelligent agent at the current moment by using a fuzzy logic system, and compare the output of each intelligent agent at the current moment with a preset threshold range to determine the state of the intelligent agent; the state of the intelligent agent includes good, average and poor; It is determined whether the proportion of the number of multi-agents determined to be in a bad state is greater than the first preset proportion; if it is greater, then based on the originally constructed auxiliary function, the first preset coefficient is selected to adjust the preset prescribed time to extend the preset prescribed time, and the adjusted auxiliary function is obtained, and the formula is: in, is the first preset coefficient; It is determined whether the proportion of the number of multi-agents determined to be in a good state is greater than the second preset proportion; if it is greater, then based on the originally constructed auxiliary function, the second preset coefficient is selected to adjust the preset prescribed time to shorten the preset prescribed time, and the adjusted auxiliary function is obtained, and the formula is: in, is the second preset coefficient.

[0021] By adopting the above scheme, the real-time output is compared with the preset threshold to judge the working status of each agent, and then the preset time in the preset time formation control of the multi-agent system is dynamically adjusted to improve the adaptability and robustness of the system.

[0022] Preferably, the step S41 further includes: Collect current environmental data; match a preset threshold range from a preset output threshold library according to the collected environmental data, and use the matched preset threshold range as the preset threshold range for comparison with the output of each intelligent agent; the preset output threshold library stores preset threshold ranges under different environmental data conditions.

[0023] By adopting the above scheme, the preset threshold range is dynamically adjusted according to the real-time output of the intelligent agent under different environmental conditions, so as to more accurately determine the working status of the intelligent agent.

[0024] Preferably, constructing the formation error equation in step S33 further includes: Determine a formation scenario of target formation control, wherein the formation scenario includes: a tight formation and a loose formation; The formation scenario controlled according to the determined target formation is matched with a preset formation error coefficient weight combination; the tight formation matches the preset formation error coefficient weight combination as a first weight combination, and the loose formation matches the preset formation error coefficient weight combination as a second weight combination; wherein, compared with the second weight combination, the ratio of the weight corresponding to the tracking error between the leader and the intelligent agent and the weight corresponding to the tracking error between the intelligent agents in the first weight combination is greater; The weight of the constructed formation error equation is assigned using the matched preset formation error coefficient weight combination.

[0025] By adopting the above scheme, the weights in the formation error equation can be flexibly adjusted according to different formation scenarios to improve the accuracy and stability of formation control.

[0026] In summary, this application has the following beneficial effects: By constructing a fuzzy logic system to handle uncertainty, it is used to evaluate the accuracy of the agent's state; a specified time control law is formulated to ensure that the multi-agent system reaches the target formation within the predetermined time; combined with a reinforcement learning algorithm, the optimal collaboration strategy between autonomous learning agents is completed; a robust control strategy is designed to enable the formation to suppress interference during movement, thereby enabling the multi-agent system to complete precise formation control within a specified time in a complex environment while having interference suppression performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of the method described in a specific embodiment; Figure 2 is a flowchart of step 1 in the method described in a specific embodiment; Figure 3 is a flow chart of step 2 in the method described in a specific embodiment; Figure 4 is a flowchart of step 3 in the method described in a specific embodiment; Figure 5 is a flow chart of step 4 in the method described in a specific embodiment; Figure 6 is a flow chart of step 5 in the method described in a specific embodiment; Figure 7 Flow chart of step 6 in the method described in the specific embodiment. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] In the prior art, it is difficult for multi-agent systems to complete formation control within a preset time, especially when facing complex environments and uncertain factors. Figure 1 As shown, the embodiment of the present application discloses a method for controlling a multi-agent system with preset time formation based on fuzzy reinforcement learning, so as to achieve the effect of realizing efficient and stable formation control of the multi-agent system in a complex environment; the specific steps include: S1. Based on graph theory, establish a communication network among multiple agents.

[0030] Through the graph theory method, a communication network of a multi-agent system is constructed to ensure smooth information transmission between each agent. Figure 2 As shown, the specific steps include: S11. Define vertex set , Represents the number of agents.

[0031] S12. Define edge set , Represents the communication connection between agents.

[0032] S13. Define the adjacency matrix , Representing an Agent With Agent When there is a communication connection between ,otherwise, ; Neighbor node set express.

[0033] S14. Define an undirected connected graph. Characterize the communication network topology of a multi-agent system; where , represents the Laplacian matrix between the agent and the leader; represents the communication matrix between the agent and the leader, where, assuming that at least one agent is connected to the leader, that is .

[0034] S2. Construct a fuzzy logic system.

[0035] The fuzzy logic system is used to calculate the output of multiple agents in real time and evaluate the accuracy of the state of multiple agents in order to cope with changes in complex environments and reduce uncertainty. Figure 3 As shown, the construction steps are as follows: S21. Establish a fuzzy rule base.

[0036] Specifically, define the IF-THEN rule in the fuzzy rule base. The rule form is: yes , yes , yes ,but yes ;in, Indicates input, Indicates the output, represents the number of fuzzy rules; Representing fuzzy sets , express The membership function of S22, using single-state fuzzification, product reasoning and central average defuzzification to calculate output ; The calculation formula includes: in, is the total number of fuzzy rules; satisfy ,make in, and , It is expressed as: .

[0037] S3. Build a multi-agent system.

[0038] In order to realize the robust formation control of multi-agents, the most important thing is to build a mathematical model of the multi-agent system, including the state equation, formation error equation and performance index function. Figure 4 As shown, the specific construction steps include: S31. Construct the state equation of the multi-agent system. The formula is: in, Indicates the system status. Indicates the control output, represents an unknown continuous nonlinear function, Indicates external interference.

[0039] S32. Construct coordinate transformation equation: in, is the tracking error, For the leader status, Represents leaders and agents The relative position of , .

[0040] S33. Construct the formation error equation, the formula is: in, , if there is a protocol , so that for , is a stable time, and is the expected accuracy; represents the adjacency matrix of the ith follower agent; represents the communication weight between the ith agent and its neighboring agents; Represents the communication connection weight between the leader and the i-th agent.

[0041] S34. Construct a performance indicator function, the formula is: in, represents the discount factor, and represents a symmetric positive definite matrix; for a multi-agent system, if is continuous, , In the collection On stable, is finite, then Admission Control Protocol , expressed as .

[0042] S4. With the goal of enabling multiple agents to complete target formation control within a preset time, construct auxiliary functions and error transformation functions.

[0043] like Figure 5 As shown, the specific steps include: S41. In order to realize the preset time control performance of the multi-agent system, construct an auxiliary function.

[0044] Construct auxiliary function M: in, is the design parameter, is the specified time; M is is strictly decreasing, and when hour, ,and ; M is smooth, M and exist All times are bounded; S42. Constructing error transformation function , The construction is as follows: in, is a constant, and .

[0045] S5. Define the optimal formation control theory and obtain the optimal control input based on the optimal formation control theory.

[0046] like Figure 6 As shown, the specific steps include: S51, updating the performance index function according to the error transformation function, which is further expressed as follows after the update: in, pass Expressed as , and Substitute the proceeds.

[0047] S52, define the optimal group control, that is, take the optimal group control , and get the optimal performance index function: ; S53, calculate the time derivative of the optimal performance index function and obtain the HJB equation: in, , , ,and is the only solution to the HJB equation. When , the optimal control input is obtained: ; S54, adjust and optimize the optimal control input. In order to achieve the best group control, Split into: in, is a design parameter, , ; in, , , ,Will Substitution , and obtain the optimal control input: ; Among them, the unknown parameters and is continuous, for and ,exist and So that: in, and All represent the optimal parameter matrix; and is the fuzzy rule number; and represents the fuzzy basis function vector; approximation error and Corresponding satisfaction and ; and is a constant.

[0048] S6. Apply the ACI structure and combine the fuzzy logic system to design identifiers, critics, actors and corresponding update rules to complete the parameter matrix update of the optimal formation control.

[0049] like Figure 7 As shown, the specific steps include: S61. Apply the ACI structure and design the identifier as follows: in, and They represent the output of FLS and identifier parameter matrix respectively, and the design identifier update rule is: in, represents the design parameters, represents a positive definite matrix; S62. Design a critic to evaluate control performance, the formula is: in, is the critic parameter matrix; the critic update rule is designed as: in, and is the design parameter; S63. Design actors to achieve control behavior. The formula is: in, Represents the actor parameter matrix, and the actor update rule is designed as: in, is the design parameter; S64, if If the value of the feature vector falls on 0, the training is terminated.

[0050] The above S1-S6 can be used to achieve precise control of the preset time formation, with priority setting , and initialize , set parameters , using S4-S6, calculate , combined with the determined fuzzy membership and fuzzy basis function vector, the calculation is obtained , and update .

[0051] A specific embodiment is different from the above embodiment in that a monitoring and adjustment mechanism for the agent state is added to further improve the adaptability of the multi-agent system in complex environments and optimize the time and resource utilization in the control process. Specifically, the method also includes: The step S41 further includes: The output of each agent at the current moment is calculated by using a fuzzy logic system, and the state of the agent is determined by comparing the output of each agent at the current moment with a preset threshold range; the state of the agent includes good, average and poor; the preset threshold range can be specifically determined based on the statistics of the real-time output range of the historical agent in different states; It is determined whether the proportion of the number of multi-agents determined to be in a poor state is greater than the first preset proportion; if it is greater, it indicates that the state of most multi-agents at the current moment is poor, and the difficulty coefficient of achieving precise control of the formation within the preset prescribed time is increased. The preset time can be extended to ensure the safety and stability of the formation, and a dynamic time adjustment factor is designed to adjust the preset prescribed time, that is, on the basis of the originally constructed auxiliary function, the first preset coefficient is selected to adjust the preset prescribed time to extend the preset prescribed time, and the adjusted auxiliary function is obtained, which is recorded as the first auxiliary function, and the formula is: in, It is the first preset coefficient, which can be set manually; It is determined whether the proportion of the number of multi-agents determined to be in good condition is greater than the second preset proportion; if it is greater, it indicates that the state of most multi-agents at the current moment is good, and the difficulty coefficient of precise control of completing the formation within the preset prescribed time is relatively low. The preset time can be shortened to improve the efficiency of the formation, and a dynamic time adjustment factor is designed to adjust the preset prescribed time. That is, based on the original constructed auxiliary function, the second preset coefficient is selected to adjust the preset prescribed time to shorten the preset prescribed time, and the adjusted auxiliary function is obtained, which is recorded as the second auxiliary function, and the formula is: in, It is the second preset coefficient and can be set manually.

[0052] In addition, considering that in the process of determining the real-time output range of the historical intelligent agent in different states, environmental factors have an impact on the real-time output range in different states, in order to further improve the adaptive ability and formation accuracy of the multi-agent system, the S41 step also includes: Sensors can be used to collect current environmental data; a preset threshold range is matched from a preset output threshold library based on the collected environmental data, and the matched preset threshold range is used as the preset threshold range for comparison with the output of each intelligent agent; the preset output threshold library stores preset threshold ranges under different environmental data conditions.

[0053] In a specific embodiment, in order to meet different requirements for formation control in different application scenarios and improve the overall formation effect, the method further includes: constructing the formation error equation in step S33 also includes: Determine a formation scenario of target formation control, wherein the formation scenario includes: a tight formation and a loose formation; The formation scenario controlled according to the determined target formation matches the preset formation error coefficient weight combination; the close formation matches the preset formation error coefficient weight combination as a first weight combination, such as: ; The loose formation matches the preset formation error coefficient weight combination as the second weight combination, such as: ; Considering that for a tight formation, the weights of the tracking errors between the leader and the agent and between agents are increased to ensure that the distance between the members of the formation is more compact and reduce the error accumulation; for a loose formation, these weights are appropriately reduced to make the formation more relaxed and adapt to larger space requirements; therefore, compared with the second weight combination, the ratio of the weight corresponding to the tracking error between the leader and the agent and the weight corresponding to the tracking error between agents in the first weight combination is set to be larger; The weight of the constructed formation error equation is assigned using a matching preset formation error coefficient weight combination.

[0054] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

Claims

1. A preset time formation control method for a multi-agent system based on fuzzy reinforcement learning, characterized in that: include: S1. Based on graph theory, establish a communication network between multiple agents; S2, build fuzzy logic system; S3. Building a multi-agent system, including: building a state equation of the multi-agent, building a formation error equation based on the multi-agent state equation, and designing a performance indicator function that introduces the formation error; S4, with the goal of multi-agents completing target formation control within a preset time, construct auxiliary functions and error transformation functions; S5. defining an optimal formation control theory, including: defining an optimal performance index function after updating the performance index function using the error change function, and introducing the HJB equation to solve the optimal control input; S6. Apply the ACI structure and combine the fuzzy logic system to design identifiers, critics, actors and corresponding update rules to complete the parameter matrix update of the optimal formation control.

2. The method for controlling a multi-agent system with preset time formation based on fuzzy reinforcement learning according to claim 1, characterized in that: The S1 step specifically includes: S11. Define vertex set , Indicates the number of agents; S12. Define edge set , Represents the communication connection between agents; S13. Define the adjacency matrix , Representing an Agent With Agent When there is a communication connection between ,otherwise, ; S14. Define an undirected connected graph , Characterize the communication network topology of a multi-agent system; where , represents the Laplacian matrix between the agent and the leader; represents the communication matrix between the agent and the leader, where it is assumed that at least one agent is connected to the leader, that is, .

3. The preset time formation control method of a multi-agent system based on fuzzy reinforcement learning according to claim 1, characterized in that: The S2 step specifically includes: S21, establish a fuzzy rule base; the rule form in the fuzzy rule base is: yes , yes , yes ,but yes ;in, Indicates input, Indicates the output, represents the number of fuzzy rules; Representing fuzzy sets , express The membership function of S22, using single-state fuzzification, product reasoning and central average defuzzification to calculate output ; The calculation formula includes: in, is the total number of fuzzy rules; satisfy ,make in, and , It is expressed as: 。 4. The method for controlling a multi-agent system with preset time formation based on fuzzy reinforcement learning according to claim 3 is characterized in that: The S3 step includes: S31. Construct the state equation of the multi-agent system. The formula is: in, Indicates the system status. Indicates the control output, represents an unknown continuous nonlinear function, Indicates external interference; S32. Construct coordinate transformation equation: in, is the tracking error, For the leader status, Represents leaders and agents The relative position of S33. Construct the formation error equation, the formula is: in, , if there is a protocol , so that for , is a stable time, and is the expected accuracy; represents the adjacency matrix of the i-th follower agent; represents the communication weight between the ith agent and its neighboring agents; represents the communication connection weight between the leader and the i-th agent; S34. Construct a performance indicator function, the formula is: in, represents the discount factor, and represents a symmetric positive definite matrix; for a multi-agent system, if is continuous, , In the collection On stable, is finite, then Admission Control Protocol , expressed as .

5. The method for controlling a multi-agent system with preset time formation based on fuzzy reinforcement learning according to claim 4, characterized in that: The S4 step includes: S41. In order to realize the preset time control performance of the multi-agent system, construct an auxiliary function M: in, is the design parameter, is the specified time; M is is strictly decreasing, and when hour, ,and ; M is smooth, M is All times are bounded; S42. Constructing error transformation function , The construction is as follows: in, is a constant, and .

6. The preset time formation control method of a multi-agent system based on fuzzy reinforcement learning according to claim 5 is characterized in that: The S5 step includes: S51, updating the performance index function according to the error transformation function, which is further expressed as follows after the update: in, pass Expressed as , and Substitute income; S52, optimal group control , and get the optimal performance index function: ; S53, calculate the time derivative of the optimal performance index function and obtain the HJB equation: in, , , ,and is the only solution to the HJB equation. When , the optimal control input is obtained: ; S54. In order to achieve optimal group control, Split into: in, is a design parameter, , ; in, , , ,Will Substitution , and obtain the optimal control input: ; Among them, the unknown parameters and is continuous, for and ,exist and So that: in, and All represent the optimal parameter matrix; and is the fuzzy rule number; and represents the fuzzy basis function vector; approximation error and Corresponding satisfaction and ; and is a constant.

7. The preset time formation control method of a multi-agent system based on fuzzy reinforcement learning according to claim 1, characterized in that: The S6 step includes: S61. Apply the ACI structure and design the identifier as follows: in, and They represent the output of FLS and identifier parameter matrix respectively, and the design identifier update rule is: in, represents the design parameters, represents a positive definite matrix; S62. Design a critic to evaluate control performance, the formula is: in, is the critic parameter matrix; the critic update rule is designed as: in, and is the design parameter; S63. Design actors to achieve control behavior. The formula is: in, Represents the actor parameter matrix, and the actor update rule is designed as: in, is the design parameter; S64, if If the value of the feature vector falls on 0, the training is terminated.

8. The method for controlling a multi-agent system with preset time formation based on fuzzy reinforcement learning according to claim 5, characterized in that: The step S41 further includes: Calculate the output of each intelligent agent at the current moment by using a fuzzy logic system, and compare the output of each intelligent agent at the current moment with a preset threshold range to determine the state of the intelligent agent; the state of the intelligent agent includes good, average and poor; It is determined whether the proportion of the number of multi-agents determined to be in a bad state is greater than the first preset proportion; if it is greater, then based on the originally constructed auxiliary function, the first preset coefficient is selected to adjust the preset prescribed time to extend the preset prescribed time, and the adjusted auxiliary function is obtained, and the formula is: in, is the first preset coefficient; It is determined whether the proportion of the number of multi-agents determined to be in a good state is greater than the second preset proportion; if greater, then based on the originally constructed auxiliary function, the second preset coefficient is selected to adjust the preset prescribed time to shorten the preset prescribed time, and the adjusted auxiliary function is obtained, and the formula is: in, is the second preset coefficient.

9. The method for controlling a multi-agent system with preset time formation based on fuzzy reinforcement learning according to claim 8, characterized in that: The step S41 further includes: Collect current environmental data; match a preset threshold range from a preset output threshold library according to the collected environmental data, and use the matched preset threshold range as the preset threshold range for comparison with the output of each intelligent agent; the preset output threshold library stores preset threshold ranges under different environmental data conditions.

10. The method for controlling a multi-agent system with preset time formation based on fuzzy reinforcement learning according to claim 4, characterized in that: The step S33 of constructing the formation error equation also includes: Determine a formation scenario of target formation control, wherein the formation scenario includes: a tight formation and a loose formation; The formation scenario controlled according to the determined target formation is matched with a preset formation error coefficient weight combination; the tight formation matches the preset formation error coefficient weight combination as a first weight combination, and the loose formation matches the preset formation error coefficient weight combination as a second weight combination; wherein, compared with the second weight combination, the ratio of the weight corresponding to the tracking error between the leader and the intelligent agent and the weight corresponding to the tracking error between the intelligent agents in the first weight combination is greater; The weight of the constructed formation error equation is assigned using a matching preset formation error coefficient weight combination.

Images