Collision avoidance learning control method applied to unmanned system cluster

Abstract

The application provides a collision avoidance learning control method applied to a unmanned system cluster, which can be applied to the field of artificial intelligence technology. The collision avoidance learning control method comprises the following steps: determining a state vector of each unmanned system in the unmanned system cluster according to a real-time position of each unmanned system; obtaining a formation error of each unmanned system according to a communication weight between the unmanned systems, the state vector of each unmanned system and an expected formation vector input error dynamics model of each follower unmanned system relative to a leading unmanned system; inputting the formation error into an evaluation neural network for each unmanned system to obtain a predicted control parameter; determining a comprehensive obstacle function value based on the real-time position of each unmanned system and a real-time position of an obstacle; and correcting the predicted control parameter based on the comprehensive obstacle function value to obtain a target control parameter, so as to control the unmanned system cluster to perform formation movement by using the target control parameter.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and more particularly to a collision avoidance learning control method applied to a swarm of unmanned systems. BACKGROUND

[0002] The swarm of unmanned systems composed of unmanned aerial vehicles and / or unmanned vehicles has been widely concerned and applied. Such a swarm usually performs tasks in complex environments, but there are various obstacles in the working environment. Therefore, the swarm of unmanned systems not only needs to effectively avoid collision, but also needs to cooperatively complete the intended task.

[0003] The formation collision avoidance method for the swarm of unmanned systems in the related art is still based on the artificial potential field method, and the chattering phenomenon may occur. The chattering phenomenon will reduce the formation efficiency of the whole swarm of unmanned systems and increase the collision risk. SUMMARY

[0004] In view of the above problems, the present application provides a collision avoidance learning control method applied to a swarm of unmanned systems.

[0005] According to a first aspect of the present application, a collision avoidance learning control method applied to a swarm of unmanned systems is provided, comprising: determining a state vector of each unmanned system in the swarm of unmanned systems according to a real-time position of each unmanned system, wherein the unmanned systems include a leader unmanned system and a plurality of follower unmanned systems; inputting a communication weight between the unmanned systems, the state vector of each unmanned system, and an expected formation vector error dynamics model of each follower unmanned system relative to the leader unmanned system, to obtain a formation error of each unmanned system in the swarm of unmanned systems, wherein the communication weight represents the communication correlation degree between the unmanned systems; inputting the formation error into an evaluation neural network for each unmanned system to obtain a predicted control parameter; determining a comprehensive obstacle function value based on the real-time position of each unmanned system and the real-time position of an obstacle, wherein the comprehensive obstacle function value is used to adjust the formation of the swarm of unmanned systems under the condition of meeting a safety constraint; correcting the predicted control parameter based on the comprehensive obstacle function value to obtain a target control parameter, so as to control the swarm of unmanned systems to perform formation movement by using the target control parameter.

[0006] According to an embodiment of the present application, the state vector of each unmanned system in the unmanned system cluster is determined according to the real-time position of each unmanned system, comprising: determining the state vector of the unmanned system according to the real-time position of the unmanned system based on the dynamic model of the unmanned system, wherein the dynamic model of the unmanned system is established based on the following manner: determining the m-order state of the unmanned system according to the real-time position to form the state vector of the unmanned system, wherein m is an integer greater than or equal to 1; constructing the dynamic model of the unmanned system according to the state vector, the first continuous function, the second continuous function and the target control parameter, wherein the first continuous function represents the change of the state vector of the unmanned system under the influence of no control, and the second continuous function represents the mapping relationship between the change of the state vector of the unmanned system under the influence of control and the control parameter of the unmanned system cluster.

[0007] According to an embodiment of the present application, the error dynamic model is constructed according to the communication topology graph of the unmanned system cluster, wherein the communication topology graph comprises nodes corresponding to each of the plurality of unmanned systems, edges connected between the nodes, and communication weights, and the edges represent the communication relationship between the plurality of unmanned systems.

[0008] According to an embodiment of the present application, the error dynamic model comprises an error dynamic sub-model corresponding to each of the plurality of follower unmanned systems, and the error dynamic sub-model is constructed based on the following manner: determining a gain coefficient based on the communication weight and the containment gain between the i-th follower unmanned system and the j-th follower unmanned system in the communication set, wherein i and j are respectively greater than or equal to 1 and less than N, i is not equal to j, N is the number of unmanned systems in the communication set that communicate with the i-th follower unmanned system, and the j-th follower unmanned system directly communicates with the i-th follower unmanned system; obtaining the controlled change of the state vector of the j-th follower unmanned system according to the control mapping parameter of the j-th follower unmanned system corresponding to the second continuous function and the real-time control parameter of the j-th follower unmanned system; obtaining the intermediate change of the state vector of the j-th follower unmanned system according to the sum of the uncontrolled change and the controlled change of the state vector of the j-th follower unmanned system corresponding to the first continuous function; obtaining the target change of the state vector of the j-th follower unmanned system according to the intermediate change of the state vector and the change of the desired formation vector of the j-th follower unmanned system relative to the leader unmanned system; and constructing the error dynamic sub-model of the i-th follower unmanned system according to the gain coefficient and the target change.

[0009] According to an embodiment of the present application, the determining the comprehensive obstacle function value of the unmanned system based on the real-time positions of the plurality of unmanned systems and the real-time position of the obstacle comprises: determining a first set of safety distances between the plurality of unmanned systems and the obstacle respectively and a second set of safety distances between the plurality of unmanned systems and adjacent unmanned systems in a formation position respectively based on the real-time positions of the plurality of unmanned systems and the real-time position of the obstacle; and determining the comprehensive obstacle function value of the unmanned system by performing safety constraints on the following unmanned system based on the first set of safety distances and the second set of safety distances for each unmanned system.

[0010] According to an embodiment of the present application, the determining the comprehensive obstacle function value of the unmanned system based on the safety constraints of the first set of safety distances and the second set of safety distances comprises: determining a target set of safety distances of the unmanned system according to the first set of safety distances and the second set of safety distances; fusing the safety constraint conditions in the target set of safety distances by using a comprehensive continuous obstacle function to obtain a safety index; and determining the comprehensive obstacle function value based on a smoothing parameter and the safety index.

[0011] According to an embodiment of the present application, the correcting the predicted control parameter based on the comprehensive obstacle function value to obtain a target control parameter comprises: determining a correction condition threshold value according to a dynamic change amount of the comprehensive obstacle function value and a gain adjustment term, wherein the dynamic change amount is obtained by processing a state vector of the following unmanned system by using a high-order derivative of the comprehensive obstacle function; determining that the predicted control parameter is the target control parameter in a case where the correction condition threshold value is greater than or equal to a preset correction threshold value; and correcting the predicted control parameter in a Lee derivative direction of the correction condition threshold value to obtain the target control parameter in a case where the correction condition threshold value is less than the preset correction threshold value.

[0012] According to an embodiment of the present application, the evaluation neural network comprises a formation controller, and the formation controller is configured to: for the i-th following unmanned system, determine a cost value of the i-th candidate control parameter under the i-th formation error based on a cost function according to the i-th formation error of the i-th following unmanned system and the i-th candidate control parameter; determine a real-time Bellman error based on a partial derivative of the cost value with respect to the i-th formation error and a j-th formation error change value corresponding to the j-th candidate control parameter of the j-th following unmanned system, wherein the j-th formation error change value represents a change trend of the j-th formation error corresponding to the j-th candidate control parameter; and determine that the candidate control parameter is a reference control parameter in a case where the real-time Bellman error satisfies a first preset condition.

[0013] According to an embodiment of the present application, the evaluation neural network is trained based on the following manner: determining an experience Bellman error based on historical control parameters and historical formation errors of the unmanned system swarm; and updating adaptive weights of the evaluation neural network by using a gradient descent adaptive algorithm until a target Bellman error meets a second preset condition, wherein the target Bellman error is a sum of the experience Bellman error and a real-time Bellman error.

[0014] According to an embodiment of the present application, the method further comprises: in a case where the inter-obstacle distance between the plurality of obstacles is less than the current safety distance of the current formation of the unmanned system swarm, switching the current formation to a target formation, and the target safety distance of the target formation is less than the inter-obstacle distance.

[0015] The second aspect of the present application provides a collision avoidance learning control device applied to an unmanned system swarm, comprising: a first determination module configured to determine a state vector of each unmanned system in the unmanned system swarm according to a real-time position of each unmanned system, wherein the plurality of unmanned systems comprises a leading unmanned system and a plurality of follower unmanned systems; a first input module configured to input an error dynamics model according to a communication weight between the plurality of unmanned systems, the state vector of each unmanned system, and an expected formation vector of each follower unmanned system relative to the leading unmanned system, to obtain a formation error of each unmanned system in the unmanned system swarm, wherein the communication weight represents a communication correlation degree between the plurality of unmanned systems; a second input module configured to input the formation error into an evaluation neural network to obtain a predicted control parameter for each unmanned system; a second determination module configured to determine a comprehensive obstacle function value based on the real-time position of each unmanned system and a real-time position of an obstacle, wherein the comprehensive obstacle function value is used to adjust the formation of the unmanned system swarm under the condition that a safety constraint condition is met; and a correction module configured to correct the predicted control parameter based on the comprehensive obstacle function value to obtain a target control parameter, so as to control the unmanned system swarm to perform formation movement by using the target control parameter.

[0016] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0017] The fourth aspect of the present application further provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the steps of the above method.

[0018] The fifth aspect of the present application further provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the steps of the above method.

[0019] According to the embodiment of the present application, the state vectors of the plurality of unmanned systems are determined according to the real-time positions of the plurality of unmanned systems in the unmanned system cluster; in order to make the formation error more consistent with the actual information interaction capability of the unmanned system cluster, the formation error of each unmanned system in the unmanned system cluster is obtained by inputting the error dynamics model of the formation vector of each follower unmanned system relative to the leading unmanned system according to the communication weight between the plurality of unmanned systems, the state vector of each unmanned system and the expected formation vector of each follower unmanned system relative to the leading unmanned system, so that the formation error is more accurate.

[0020] The evaluation neural network is used to predict the control parameters of each unmanned system respectively, and the predicted control parameters are obtained. Since the comprehensive barrier function value can adjust the formation of the unmanned system cluster under the condition of meeting the safety constraint condition, the predicted control parameters are corrected based on the comprehensive barrier function value, so that the target control parameters which can not only avoid obstacles but also maintain the formation are obtained, which not only reduces the chattering when approaching the obstacle, but also automatically balances the "obstacle avoidance safety" and "formation efficiency". BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments of the present application taken in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A scenario diagram of application of the collision avoidance learning control method applied to the unmanned system cluster according to the embodiment of the present application is shown.

[0023] Figure 2 A flowchart of the collision avoidance learning control method applied to the unmanned system cluster according to the embodiment of the present application is shown.

[0024] Figure 3 A schematic diagram of the expected formation of the unmanned vehicle cluster according to the embodiment of the present application is shown.

[0025] Figure 4 A trajectory diagram of the collision avoidance learning control method applied to the unmanned system cluster according to the embodiment of the present application is shown.

[0026] Figure 5 A trajectory diagram based on the artificial potential field method according to the embodiment of the present application is shown.

[0027] Figure 6 A formation control collision avoidance learning control system applied to the unmanned cluster system according to the embodiment of the present application is shown.

[0028] Figure 7 A structure block diagram of the collision avoidance learning control device applied to the unmanned system cluster according to the embodiment of the present application is shown.

[0029] Figure 8A block diagram of an electronic device suitable for implementing the collision avoidance learning control method applied to a swarm of unmanned systems according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0030] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It is to be understood, however, that the description is merely exemplary of the present application, and is not intended to limit the scope of the present application. In the following detailed description of the embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present application.

[0031] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the term "includes" and tautological expressions thereof, such as "including," "includes," "include," "contains," "containing," and so forth, shall not be taken to exclude

[0032] All terms used herein including technical and scientific terms have the same meanings as commonly understood by one of ordinary skill in the art unless otherwise defined herein. It should be noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification, and not be interpreted in an idealized or overly formal way.

[0033] In the case where expressions similar to "at least one of A, B, and C, and the like" are used, it is generally to be understood that the expression is to be interpreted to include at least one of A, B, or C, or any combination of the items A, B, and C (for example, the system "having at least one of A, B, and C" shall include a system "having A alone," a system "having B alone," a system "having C alone," a system "having A and B together," a system "having A and C together," a system "having B and C together," and / or a system "having A, B, and C together," etc.).

[0034] A swarm of unmanned systems composed of unmanned aerial vehicles and unmanned ground vehicles has been widely concerned and applied, such a swarm usually performs a task in a complex environment, but there are various obstacles in the working environment. Therefore, the swarm of unmanned systems not only effectively avoids collision, but also cooperatively completes the intended task.

[0035] The formation collision avoidance method for the swarm of unmanned systems in the related art is still based on the artificial potential field method, and the chattering phenomenon may occur, which reduces the formation efficiency of the entire swarm of unmanned systems and increases the collision risk.

[0036] According to the application, an application scene diagram of the collision avoidance learning control method applied to the unmanned system cluster is shown.

[0037] Figure 1 An application scene diagram of the collision avoidance learning control method applied to the unmanned system cluster is shown.

[0038] As shown in Figure 1 The application scene 100 according to the embodiment can include an unmanned system cluster 110, a network 120 and a server 130. The unmanned system cluster 110 can include a first unmanned system 101, a second unmanned system 102 and a third unmanned system 103. The first unmanned system 101 can be a leading unmanned system, and the second unmanned system 102 and the third unmanned system 103 can be follower unmanned systems.

[0039] The network 120 is a medium for providing a communication link between the unmanned system cluster 110 and the server 130. The network 120 can include various connection types, such as a wireless communication link.

[0040] The first unmanned system 101, the second unmanned system 102 and the third unmanned system 103 interact with the server 130 through the network 120, so that the unmanned system cluster sends real-time positions of the unmanned systems to the server 130, and the server 130 sends control parameters to the unmanned systems. For example, the first unmanned system 101, the second unmanned system 102 and the third unmanned system 103 can receive the control parameters sent by the server, so that the unmanned systems perform formation movement according to the control parameters (only an example).

[0041] The server 130 can be a server providing various services, such as predicting and correcting control parameters for the formation of the first unmanned system 101, the second unmanned system 102, and the third unmanned system 103, for example.

[0042] It should be noted that the collision avoidance learning control method for the unmanned system cluster provided by the embodiments of the present application can be executed by the server 130.

[0043] It should be understood that Figure 1 The number of unmanned systems, networks, and servers in the above description is only illustrative. Any number of unmanned systems, networks, and servers can be provided according to the needs of implementation.

[0044] Figure 2 A flowchart of the collision avoidance learning control method for the unmanned system cluster according to the embodiments of the present application is shown.

[0045] As Figure 2 The collision avoidance learning control method for the unmanned system cluster of this embodiment includes operations S210-S250, which can be executed by a server.

[0046] At operation S210, a state vector of each of a plurality of unmanned systems is determined according to a real-time position of each of the plurality of unmanned systems in the unmanned system cluster, wherein the plurality of unmanned systems includes a leading unmanned system and a plurality of follower unmanned systems.

[0047] According to the embodiments of the present application, the unmanned system can be a drone, an unmanned vehicle, or the like. The unmanned system cluster can be composed of a plurality of drones or a plurality of unmanned vehicles, or can be composed of at least one drone and at least one unmanned vehicle.

[0048] According to the embodiments of the present application, the leading unmanned system can dynamically adjust the formation of the unmanned system cluster according to the external environment (such as obstacles, task targets), and send instructions to the follower unmanned systems. The server executing the method can be provided in the leading unmanned system.

[0049] According to the embodiments of the present application, the follower unmanned system can execute a local action by receiving the instructions of the leading unmanned system. The local action can be maintaining the formation, obstacle avoidance, etc. And feedback the action execution result to the leading unmanned system.

[0050] According to the embodiments of the present application, the state vector can be a mathematical vector representing the current state of the unmanned system. The state vector can include information such as the three-dimensional real-time position, velocity, acceleration, etc. of the unmanned system.

[0051] In operation S220, a formation error of each unmanned system in the unmanned system swarm is obtained by inputting the communication weight between the unmanned systems, the state vector of each unmanned system and the desired formation vector error of each follower unmanned system relative to the leader unmanned system into the error dynamics model.

[0052] According to an embodiment of the present application, the follower unmanned systems can communicate with the leader unmanned system, but there can be a situation that the follower unmanned systems cannot communicate with each other. In order to make the formation error more consistent with the actual information interaction capability of the unmanned system swarm, the communication association degree between the unmanned systems needs to be considered. If the information is out of communication, the formation error is calculated based on the "state information of the neighbor unmanned system that cannot be obtained", which will lead to invalid control strategy.

[0053] According to an embodiment of the present application, the desired formation vector of the follower unmanned system relative to the leader unmanned system can be a mathematical vector for representing the desired relative position or pose, i.e., defining the position or pose of the follower unmanned system in the leader unmanned system reference system.

[0054] According to an embodiment of the present application, the error dynamics model can be a mathematical model for representing the change of the formation error with time.

[0055] According to an embodiment of the present application, the formation error can be the deviation between the actual formation vector and the desired formation vector. The actual formation vector can be the real-time formation vector of the follower unmanned system.

[0056] In operation S230, for each unmanned system, the formation error is input into an evaluation neural network to obtain a predicted control parameter.

[0057] According to an embodiment of the present application, the evaluation neural network can include an input layer, a hidden layer and an output layer. The formation error can be input to the input layer, and if the cost of the control parameter can be considered, the control parameter can be taken as an auxiliary input. The hidden layer can convert linear input into nonlinear features through an activation function to approximate the optimal cost. The output layer can map the features of the hidden layer to the control parameter.

[0058] In operation S240, based on the real-time positions of the unmanned systems and the real-time positions of the obstacles, a comprehensive obstacle function value is determined, wherein the comprehensive obstacle function value is used to adjust the formation of the unmanned system swarm under the condition that the safety constraint condition is met.

[0059] According to an embodiment of the present application, the obstacle can be a static or dynamic object that causes the unmanned systems in the unmanned swarm to collide. For example, the obstacle can be a tree, a building, an aircraft, etc.

[0060] According to an embodiment of the present application, the safety constraint condition can be that a safe distance is kept between the unmanned system and the obstacle.

[0061] In operation S250, the prediction control parameter is corrected based on the comprehensive obstacle function value to obtain a target control parameter, so as to control the formation movement of the unmanned system cluster by using the target control parameter.

[0062] According to an embodiment of the present application, under the condition that the prediction control parameter realizes accurate formation, in order to avoid the obstacle, the prediction control parameter is corrected based on the comprehensive obstacle function value, and the optimal path that neither collides with the obstacle nor breaks the formation is selected by balancing the 'obstacle avoidance safety' and the 'formation efficiency'.

[0063] According to an embodiment of the present application, the state vector of each unmanned system in the unmanned system cluster is determined according to the real-time position of each unmanned system in the unmanned system cluster; in order to make the formation error more consistent with the actual information interaction capability of the unmanned system cluster, the formation error of each unmanned system in the unmanned system cluster is obtained by inputting the error dynamics model of each follower unmanned system relative to the desired formation vector of the leading unmanned system according to the communication weight between the unmanned systems, the state vector of each unmanned system and each follower unmanned system, so that the formation error is more accurate.

[0064] The evaluation neural network is used to predict the control parameter of each unmanned system respectively to obtain the prediction control parameter. Since the comprehensive obstacle function value can adjust the formation of the unmanned system cluster under the condition of meeting the safety constraint condition, the prediction control parameter is corrected based on the comprehensive obstacle function value, so as to obtain the target control parameter that neither collides with the obstacle nor breaks the formation, thereby reducing the situation of chattering when approaching the obstacle, and automatically balancing the 'obstacle avoidance safety' and the 'formation efficiency'.

[0065] According to an embodiment of the present application, the state vector of each unmanned system in the unmanned system cluster is determined according to the real-time position of each unmanned system in the unmanned system cluster, and the state vector of each unmanned system is determined based on the dynamic model of the unmanned system according to the real-time position of the unmanned system, wherein the dynamic model of the unmanned system is obtained based on the following manner: the m-order state of the unmanned system is determined according to the real-time position to constitute the state vector of the unmanned system, wherein m is an integer greater than or equal to 1; the dynamic model of the unmanned system is constructed according to the state vector, the first continuous function, the second continuous function and the real-time control parameter of the unmanned system, wherein the first continuous function represents the change of the state vector of the unmanned system under the influence of no control, and the second continuous function represents the mapping relationship between the change of the state vector of the unmanned system under the influence of control and the control parameter of the unmanned system cluster.

[0066] According to an embodiment of the present application, the dynamic model of the unmanned system can be represented as:

[0067] (1);

[0068] wherein t is time, i represents the number of the follower unmanned system (i = 1, 2, …, N), N represents the number of follower unmanned systems in the cluster of unmanned systems, is the lth order state of the ith follower unmanned system, and l is an integer greater than 1 and less than m-1, is the first derivative of . is the mth order state of the ith follower unmanned system, is the first derivative of . represents the real-time position of the ith follower unmanned system, , represents the state vector of the ith follower unmanned system at time t, and is the first continuous function for the lth order state of the ith follower unmanned system, is the first continuous function for the mth order state of the ith follower unmanned system, is the second continuous function for the mth order state of the ith follower unmanned system, is the real-time control parameter of the ith follower unmanned system at time t, and the above model can also be represented as:

[0069] (2);

[0070] wherein and , and is a zero matrix of . The dynamic model of the leader unmanned system is

[0071] (3);

[0072] wherein, is the real-time position of the leader unmanned system at time t, and the number of the leader unmanned system is 0. is the first order state of the leader unmanned system at time t, is the lth order state of the leader unmanned system at time t, but the corresponding l of the leader unmanned system is an integer greater than 1 and less than m, is the first derivative of . The state vector of the leader unmanned system , and are the transpose of the first order state, the transpose of the second order state, …, and the transpose of the mth order state of the leader unmanned system, respectively. is a first continuous function for the first order state of the lead unmanned system. In addition, .

[0073] According to an embodiment of the present application, the error dynamics model is constructed according to a communication topology graph of the unmanned system cluster, wherein the communication topology graph comprises nodes corresponding to the unmanned systems respectively, edges connecting the nodes, and communication weights, the edges representing communication relationships between the unmanned systems.

[0074] According to an embodiment of the present application, a communication topology graph of the unmanned system cluster is established. The communication connection relationship between the follower unmanned systems is described by a directed graph, and the formation shape can be represented by a directed graph , wherein represents a set of nodes, the nodes representing the follower unmanned systems in the unmanned system cluster, i.e. respectively represent a node corresponding to the first follower unmanned system, a node corresponding to the second follower unmanned system, and a node corresponding to the Nth follower unmanned system. It should be noted that the lead unmanned system can communicate with the follower unmanned systems.

[0075] represents a set of edges, the edges representing communication relationships between the follower unmanned systems. represents a weight connection matrix, represents a communication weight between the ith follower unmanned system and the jth follower unmanned system. Wherein represents that the ith follower unmanned system can receive information of the jth follower unmanned system, and , represents a communication set composed of follower unmanned systems capable of communicating with the ith follower unmanned system, represents an in-degree matrix of the follower unmanned systems, wherein , the Laplacian matrix of the follower unmanned systems is , wherein , .

[0076] According to an embodiment of the present application, the error dynamics model comprises a plurality of error dynamics sub-models corresponding to the respective follower unmanned systems, and the error dynamics sub-models are constructed based on the following manner: determining a gain coefficient based on a communication weight and a containment gain between the ith follower unmanned system and the jth follower unmanned system in the communication set, where i and j are respectively greater than or equal to 1 and less than N, i is not equal to j, N is the number of unmanned systems in the communication set that communicate with the ith follower unmanned system, and the jth follower unmanned system directly communicates with the ith follower unmanned system; obtaining a controlled change of a state vector of the jth follower unmanned system according to a control mapping parameter of the jth follower unmanned system corresponding to a second continuous function and a real-time control parameter of the jth follower unmanned system; obtaining an intermediate change of the state vector of the jth follower unmanned system according to a sum of an uncontrolled change of the state vector of the jth follower unmanned system corresponding to a first continuous function and the controlled change; obtaining a target change of the state vector of the jth follower unmanned system according to the intermediate change of the state vector and a change of an expected formation vector of the jth follower unmanned system relative to the leader unmanned system; and constructing an error dynamics sub-model of the ith follower unmanned system according to the gain coefficient and the target change.

[0077] According to an embodiment of the present application, the ith formation error is constructed based on a communication topology graph of the unmanned swarm system.

[0078] (4);

[0079] where i and j are respectively greater than or equal to 1 and less than N, i is not equal to j, N i is the number of unmanned systems in the communication set that communicate with the ith follower unmanned system , N i is the number of unmanned systems in the communication set that communicate with the ith follower unmanned system indicates an expected formation vector of the ith follower unmanned system relative to the leader unmanned system, indicates an expected formation vector of the jth follower unmanned system relative to the leader unmanned system, indicates a state vector of the ith follower unmanned system, indicates a state vector of the jth follower unmanned system, is a state vector of the leader unmanned system. , is a containment gain for the ith follower unmanned system. Thus, the error dynamics sub-model corresponding to the ith follower unmanned system is:

[0080] (5);

[0081] where, , represents a communication weight between the i-th follower unmanned system and the j-th follower unmanned system, is a gain coefficient.

[0082] represents a control mapping parameter of the j-th follower unmanned system corresponding to the second continuous function, is a real-time control parameter of the j-th follower unmanned system, is a product between and is an uncontrollable change amount of the state vector of the j-th follower unmanned system corresponding to the first continuous function, is a change amount of the desired formation vector of the j-th follower unmanned system relative to the leader unmanned system, is a target change amount of the state vector of the j-th follower unmanned system. Wherein, , is an uncontrollable change amount of the state vector of the i-th follower unmanned system corresponding to the first continuous function, or is a self-restraint gain of the i-th follower unmanned system, is a restraint gain of the i-th follower unmanned system relative to the j-th follower unmanned system, is an extended set containing the i-th follower unmanned system and the communication set, , .

[0083] According to an embodiment of the present application, the evaluation neural network comprises a formation controller, and the formation controller is configured to: for the i-th follower unmanned system, determine, based on a cost function, a cost value of the i-th candidate control parameter under the i-th formation error according to the i-th formation error of the i-th follower unmanned system and the i-th candidate control parameter, determine a real-time Bellman error based on a partial derivative of the cost value with respect to the i-th formation error and a j-th formation error change value corresponding to the j-th candidate control parameter of the j-th follower unmanned system, wherein the j-th formation error change value represents a change trend of the j-th formation error corresponding to the j-th candidate control parameter, and determine that the candidate control parameter is a reference control parameter when the real-time Bellman error satisfies a first preset condition.

[0084] In order to realize formation, for the i-th follower unmanned system, a cost function is constructed as follows

[0085] (6);

[0086] , , and ​are different symmetric and positive definite matrices, is a real-time control parameter of the jth follower unmanned system, is a real-time control parameter of the ith follower unmanned system. The ith formation error and the ith candidate control parameter of the ith follower unmanned system are brought into the cost function to determine the cost value of the ith candidate control parameter under the ith formation error.

[0087] According to the above cost function, the optimal formation control strategy is obtained by using the Bellman optimality condition as:

[0088] (7) ;

[0089] wherein d i represents the indegree of the ith follower unmanned system, is a self-restraint gain of the ith follower unmanned system, , is a second continuous function for the mth order state of the ith follower unmanned system. is the cost value is a partial derivative of the ith formation error . is an inverse matrix of .

[0090] After the ith candidate control parameter and the jth candidate control parameter are brought into the following Bellman error equation (formula (8)), the reference control parameter is determined when the real-time Bellman error is equal to 0. The preset condition can be that the real-time Bellman error is equal to 0.

[0091] (8) ;

[0092] represents a jth formation error change value corresponding to the jth candidate control parameter of the jth follower unmanned system, represents a partial derivative of the cost value with respect to the ith formation error.

[0093] According to an embodiment of the present application, the evaluation neural network is trained based on the following manner: based on historical control parameters and historical formation errors of the unmanned system cluster, an experience Bellman error is determined; adaptive weights of the evaluation neural network are updated by using a gradient descent adaptive algorithm until a target Bellman error meets a second preset condition, wherein the target Bellman error is a sum of the experience Bellman error and a real-time Bellman error.

[0094] According to embodiments of the present invention, the evaluation neural network may include multiple single-evaluation neural networks corresponding to each of the following unmanned systems, and an online learning controller for each single-evaluation neural network is constructed. The single-evaluation neural network structure is constructed, approximating the optimal cost function, and based on the real-time position of the i-th following unmanned system, predictive control parameters for the i-th following unmanned system during the network learning process are calculated. :

[0095] (9);

[0096] in Activation function Regarding the error of the i-th formation The partial derivatives, To evaluate the weights of the network.

[0097] The real-time Bellman error is calculated using the following formula. :

[0098] (10);

[0099] According to the i-th following unmanned system at a historical moment Historical state vector And historical formation error The historical control parameters for the i-th following unmanned system during the network learning process are calculated using formula (11). .

[0100] (11);

[0101] Accelerate using experience playback technology Convergence, Empirical Bellman Error for

[0102] (12);

[0103] in Representing a historical moment and satisfying , , To be at a historical moment The evaluation network weights are used to construct the target Bellman error by combining real-time learning data and empirical data. for

[0104] (13);

[0105] in, To minimize the target Bellman error, the number of historical state vectors to be stored. Using the gradient descent adaptive method, the adaptive weight update rule for a single-evaluation neural network is determined as follows:

[0106] (14);

[0107] For learning rate, , This is the updated adaptive weight. The closer to the reference control parameters The better the training results of a single evaluation neural network.

[0108] According to an embodiment of the present invention, the online learning of the neural network iteratively optimizes the adaptive weights based on the actual collision avoidance effect, eliminating the need for manual parameter adjustment and fundamentally eliminating chattering caused by improper parameters.

[0109] According to an embodiment of the present invention, determining a comprehensive obstacle function value based on the real-time positions of multiple unmanned systems and the real-time positions of obstacles includes: determining a first set of safe distances between the multiple unmanned systems and obstacles, and a second set of safe distances between the multiple unmanned systems and their respective adjacent unmanned systems in the formation position, based on the real-time positions of the multiple unmanned systems and the real-time positions of obstacles; and for each unmanned system, applying safety constraints to the following unmanned system based on the first set of safe distances and the second set of safe distances to determine the comprehensive obstacle function value of the unmanned system.

[0110] According to an embodiment of the present invention, the first Following unmanned systems and the first First safe distance set between obstacles It can be represented as:

[0111] (15);

[0112] It is a state space China satisfies of , This represents the real-time position of the i-th following unmanned system. , , Indicates the i-th following unmanned system and the i-th The distance between obstacles Describes the shape of a hyperellipsoidal obstacle and its maximum eigenvalue. Equals 1, For the first The location of the obstacle, For the first The radius of the obstacle, To follow the radius of the unmanned system, it is noted that the radii of the plurality of unmanned systems that follow can be the same. To the first set of indicators of the obstacle.

[0113] The second set of safety distances C between the i-th follower unmanned system and its adjacent unmanned system in the formation position. iaj Can be expressed as:

[0114] (16);

[0115] Where C iaj State space satisfies The , and , d iaj Indicates the distance between the i-th follower and the adjacent unmanned system y aj Indicates the real-time position of the adjacent unmanned system . Set of indicators of the adjacent unmanned system .

[0116] According to the embodiment of the application, the unmanned system is safety-constrained based on the first set of safety distances and the second set of safety distances, the comprehensive obstacle function value of the unmanned system is determined, comprising: determining the target safety distance set of the unmanned system according to the first set of safety distances and the second set of safety distances; fusing the safety constraint conditions in the target safety distance set by using the comprehensive continuous obstacle function to obtain the safety indicators; determining the comprehensive obstacle function value based on the smoothing parameter and the safety indicators.

[0117] The target safety distance set is constructed by using Boolean logic operation :

[0118] (17);

[0119] Where, Indicates the i-th Set of indicators of the obstacle and the set of indicators of the adjacent unmanned system . The safety indicators represented by the local high-order obstacle function of the i-th follower unmanned system and the q-th constraint object (obstacle or adjacent unmanned system) are set as C

[0120] (18);

[0121] The smoothing parameter is denoted as , The larger, the more strictly all safety constraints are satisfied. is the overall barrier function value of the i-th follower UAV, and is the "total safety index" after fusing all local constraints.

[0122] According to an embodiment of the present application, the predicted control parameter is corrected based on the overall barrier function value to obtain a target control parameter, including: determining a correction condition threshold value according to a dynamic change amount of the overall barrier function value and a gain adjustment term, wherein the dynamic change amount is obtained by processing a state vector of the follower unmanned system using a high-order derivative of the overall barrier function; in the case that the correction condition threshold value is greater than or equal to a preset correction threshold value, determining that the predicted control parameter is the target control parameter; in the case that the correction condition threshold value is less than the preset correction threshold value, correcting the predicted control parameter in the Lie derivative direction of the correction condition threshold value to obtain the target control parameter.

[0123] Constructing a quadratic programming problem based on the overall barrier function

[0124] (19) ;

[0125] represents the average value of all control parameters that satisfy the safety constraint conditions in the target safety distance set. is the predicted control parameter of the i-th follower unmanned system in the network learning process. represents the safety constraint condition in the target safety distance set, , the dynamic change amount of the overall barrier function value can be . is the gain adjustment term, until the correction condition threshold value is obtained when l is equal to m. s.t. is the abbreviation of "subject to", used to introduce the constraint conditions of the optimization problem ". means that the upper bound of is greater than or equal to 0. The preset correction threshold value can be 0.

[0126] According to the KKT condition (Karush-Kuhn-Tucker), the explicit solution of the quadratic programming problem is

[0127] (20) ;

[0128] In the case that the correction condition threshold value is greater than or equal to 0, is equal to 0, the predicted control parameter is determined as the target control parameter. Otherwise, in the case that the correction condition threshold value In the case of less than 0, as shown in formula (20), the prediction control parameter is corrected along the Lie derivative direction of the correction condition threshold, The Lie derivative term is represented.

[0129] In the unmanned system cluster, the chattering phenomenon accelerates the wear and tear of the mechanical structure due to frequent start-stop or reverse movement, and even causes the parts to break. The control input of chattering presents a pulse fluctuation instead of a smooth output, and the energy consumption of the pulse fluctuation is large, which may affect the task endurance.

[0130] At the same time, chattering can cause irregular fluctuations in the position and speed of a single unmanned system, and the cluster formation relies on the state synchronization of adjacent individuals. If a single unmanned system frequently deviates from the expected position due to chattering, it will lead to the collapse of the formation of the unmanned system cluster or cause collisions between adjacent unmanned systems.

[0131] According to the embodiment of the present application, the formation error is defined based on the communication topology graph, and the formation error is calculated only by using the state vector of the neighbor unmanned system that can be obtained by the unmanned aerial vehicle, so as to ensure the credibility of the formation error and avoid invalid control that deviates from the actual information interaction capability. The error dynamics model is constructed to quantify the evolution law of the formation error over time, and provide a clear dynamic basis for the control strategy.

[0132] At the same time, the introduction of the comprehensive barrier function can fuse multiple barrier constraints, and can effectively avoid the chattering phenomenon of the artificial potential field method. The correction threshold is designed to combine the high-order derivative of the comprehensive barrier function to judge the “dynamic safety” (not only the current safety, but also the future will not approach the barrier), so as to eliminate the hidden danger of chattering at the root.

[0133] The single evaluation neural network is used to replace the optimal cost function solved by analysis, and the network weight value is updated online by minimizing the Bellman error, so that the control strategy can learn the optimal solution with small error and low energy consumption.

[0134] According to the embodiment of the present application, the above method further comprises: in the case that the barrier distance between the plurality of barriers is less than the current safety distance of the current formation of the unmanned system cluster, switching the current formation to a target formation, and the target safety distance of the target formation is less than the barrier distance.

[0135] According to the embodiment of the present application, for the formation control problem of the unmanned cluster system in a narrow environment, considering that the obstacle avoidance operation may consume a large amount of energy, an adaptive formation dynamic adjustment strategy is designed.

[0136] A candidate formation formation for the unmanned system cluster is designed in advance , and the minimum safe passing distance of each formation formation is calculated, and the minimum safe passing distance can also be called a safety distance. Among them, the safety distance of the first formation formation is .

[0137] When the unmanned cluster system is in formation movement, the inter-obstacle distance between multiple obstacles in front is detected in real time When the inter-obstacle distance is detected to be less than the required safety distance at the moment for the current formation, i.e. , a formation reconstruction mechanism is triggered. A target formation that satisfies is selected from a pre-defined formation library , The target safety distance, and the unmanned cluster system is controlled to adjust the formation shape, learn a new control strategy, realize formation transformation, and ensure safe passing.

[0138] In an embodiment, the unmanned system cluster can be an unmanned vehicle cluster, and the unmanned vehicle cluster can include multiple follower unmanned vehicles and a leader unmanned vehicle. The dynamic model of the follower unmanned vehicle is shown in equation (21).

[0139] (21) ;

[0140] where y i and represent the position of the th follower unmanned vehicle, is the first derivative of , represents the speed of the th follower unmanned vehicle, is the first derivative of . is the real-time control parameter of the th follower unmanned system, and the state vector of the th follower unmanned system is: .

[0141] For example, the initial position and speed of the th follower unmanned vehicle are selected as , and the real-time position and speed of the leader unmanned vehicle are selected as .

[0142] The system parameters are selected as , and I is an identity matrix, such as a second-order identity matrix , a fourth-order identity matrix . The activation function is selected as: . z o is the center coordinate of the obstacle, r o is the radius of the obstacle, and Eo is a positive definite symmetric matrix.

[0143] For example, the area parameters of the second obstacle are:

[0144]

[0145] The area parameters of the third and fourth obstacles are respectively:

[0146]

[0147] The area parameter of the third obstacle is:

[0148] .

[0149] The radius of the unmanned vehicle is , and the parameters of the comprehensive obstacle function are selected as .

[0150] Figure 3 A schematic diagram of a desired formation of a cluster of unmanned vehicles according to an embodiment of the present application is shown.

[0151] As shown in Figure 3 , the desired formation of the cluster of unmanned vehicles is , and the safety distance is respectively set as .

[0152] The coordinates of the leader (pilot unmanned vehicle) in the formation F1 are (0, 0), the coordinates of the follower 1 (pursuit unmanned vehicle 1) are (-4, -4), the coordinates of the follower 2 (pursuit unmanned vehicle 2) are (-4, 4), and the coordinates of the follower 3 (pursuit unmanned vehicle 3) are (-8, 0).

[0153] The coordinates of the leader (pilot unmanned vehicle) in the formation F2 are (0, 0), the coordinates of the follower 1 (pursuit unmanned vehicle 1) are (-4, 0), the coordinates of the follower 2 (pursuit unmanned vehicle 2) are (-8, 0), and the coordinates of the follower 3 (pursuit unmanned vehicle 3) are (-12, 0).

[0154] In an embodiment, when the cluster of unmanned vehicles is driving in the formation F1, but there are multiple obstacles in front of the driving, the distance between the obstacles is greater than 2 and less than 10, the formation F1 can be transformed into the formation F2 to ensure the safe passing of the cluster of unmanned vehicles.

[0155] Figure 4 A trajectory diagram of the collision avoidance learning control method applied to the cluster of unmanned systems according to an embodiment of the present application is shown.

[0156] Figure 5 A trajectory diagram based on the artificial potential field method according to an embodiment of the present application is shown.

[0157] On the basis of the above embodiments, the trajectory of the leader is x0, the trajectory of the follower 1 is x1, the trajectory of the follower 2 is x2, and the trajectory of the follower 3 is x3. The circles or ellipses in the figure represent obstacles.

[0158] Figure 4 The unmanned vehicle cluster successfully achieves formation and effectively avoids collision. Figure 5 The method is to achieve formation control with smaller oscillation and successfully avoid collision.

[0159] Figure 6 A formation control and collision avoidance learning control system applied to an unmanned cluster system according to an embodiment of the present application is shown.

[0160] As shown in Figure 6 , the formation control and collision avoidance learning control system applied to the unmanned cluster system 600 includes a detection and positioning device 610, a microprocessor 620, a first driving device 630, a second driving device 640, a memory 650, and a communication component 660.

[0161] The detection and positioning device 610 is used to provide positioning information and detection information for the unmanned system cluster, mainly for detecting the distance between the obstacles and the distance between the unmanned cluster systems, and transmitting the detection information to the microprocessor 620 for calculating the obstacle function and the formation error.

[0162] The microprocessor 620 is mainly composed of typical embedded chips and related functional circuits, used for executing neural network training and control calculation. The microprocessor 620 can be part of the server 130, and the microprocessor 620 can be installed on the lead unmanned system. The output target control parameters will be added to the first driving device 630 and the second driving device 640.

[0163] The first driving device 630 and the second driving device 640 are used to provide power for the unmanned system and are responsible for executing control instructions. The first driving device 630 and the second driving device 640 can be selected as micro motor devices, with external wheels, rotors, etc. Note that the driving device can be multiple, but for the design of the present application, the number of driving devices should be even.

[0164] The memory 650 is coupled to the microprocessor, such as a functional memory, a non-volatile storage medium, etc. Used to store the state trajectory information of the system, learning data, control signals, and pre-designed formation formation information.

[0165] The communication component 660 is used for communication between other unmanned systems. In the embodiment of the unmanned cluster system, the communication component 660 can help the unmanned cluster system to realize information sharing of strategy, error state, etc. between the unmanned cluster systems.

[0166] Figure 7 A structural block diagram of the collision avoidance learning control device applied to the swarm of unmanned systems according to an embodiment of the present application is shown.

[0167] As shown in Figure 7 the collision avoidance learning control device 700 applied to the swarm of unmanned systems of this embodiment includes a first determination module 710, a first input module 720, a second input module 730, a second determination module 740, and a correction module 750.

[0168] The first determination module 710 is configured to determine a state vector of each of the plurality of unmanned systems according to a real-time position of each of the plurality of unmanned systems, wherein the plurality of unmanned systems include a leader unmanned system and a plurality of follower unmanned systems. In an embodiment, the first determination module 710 can be configured to perform the operation S210 described above, and details are not repeated here.

[0169] The first input module 720 is configured to input an error dynamics model of formation according to a communication weight between the plurality of unmanned systems, the state vector of each of the unmanned systems, and an expected formation vector of each of the follower unmanned systems relative to the leader unmanned system, to obtain a formation error of each of the plurality of unmanned systems in the swarm of unmanned systems, wherein the communication weight represents a communication correlation degree between the plurality of unmanned systems. In an embodiment, the first input module 720 can be configured to perform the operation S220 described above, and details are not repeated here.

[0170] The second input module 730 is configured to input the formation error into an evaluation neural network for each of the unmanned systems to obtain a predicted control parameter. In an embodiment, the second input module 730 can be configured to perform the operation S230 described above, and details are not repeated here.

[0171] The second determination module 740 is configured to determine a comprehensive obstacle function value based on the real-time position of each of the plurality of unmanned systems and a real-time position of an obstacle, wherein the comprehensive obstacle function value is used to adjust the formation of the swarm of unmanned systems under the condition that a safety constraint condition is met. In an embodiment, the second determination module 740 can be configured to perform the operation S240 described above, and details are not repeated here.

[0172] The correction module 750 is configured to correct the predicted control parameter based on the comprehensive obstacle function value to obtain a target control parameter, so as to control the swarm of unmanned systems to perform formation movement by using the target control parameter. In an embodiment, the correction module 750 can be configured to perform the operation S250 described above, and details are not repeated here.

[0173] According to an embodiment of the present application, the first determining module 710 comprises a first determining sub-module. The first determining sub-module is configured to determine a state vector of the unmanned system based on a real-time position of the unmanned system according to a dynamic model of the unmanned system, wherein the dynamic model of the unmanned system is established based on the following manner: determining m orders of states of the unmanned system according to the real-time position to form the state vector of the unmanned system, wherein m is an integer greater than or equal to 1; constructing the dynamic model of the unmanned system according to the state vector, a first continuous function, a second continuous function and a target control parameter, wherein the first continuous function represents a change of the state vector of the unmanned system under no control influence, and the second continuous function represents a mapping relationship between a change amount of the state vector of the unmanned system under control influence and a control parameter of the unmanned system cluster.

[0174] According to an embodiment of the present application, the error dynamic model is constructed according to a communication topology graph of the unmanned system cluster, wherein the communication topology graph comprises nodes corresponding to the respective unmanned systems, edges connecting the nodes, and communication weights, and the edges represent communication relationships between the unmanned systems.

[0175] According to an embodiment of the present application, the error dynamic model comprises a plurality of error dynamic sub-models corresponding to the respective follower unmanned systems, and each error dynamic sub-model is constructed based on the following manner: determining a gain coefficient based on a communication weight and a containment gain between the ith follower unmanned system and the jth follower unmanned system in the communication set, wherein i and j are respectively greater than or equal to 1 and less than N, i is not equal to j, N is a number of unmanned systems in the communication set that communicate with the ith follower unmanned system, and the jth follower unmanned system directly communicates with the ith follower unmanned system; obtaining a controlled change amount of a state vector of the jth follower unmanned system according to a control mapping parameter of the jth follower unmanned system corresponding to the second continuous function and a real-time control parameter of the jth follower unmanned system; obtaining an intermediate change amount of the state vector of the jth follower unmanned system according to a sum of an uncontrolled change amount of the state vector of the jth follower unmanned system corresponding to the first continuous function and the controlled change amount; obtaining a target change amount of the state vector of the jth follower unmanned system according to the intermediate change amount and a change amount of an expected formation vector of the jth follower unmanned system relative to the leader unmanned system; and constructing an error dynamic sub-model of the ith follower unmanned system according to the gain coefficient and the target change amount.

[0176] According to an embodiment of the present application, the second determining module 740 comprises a second determining sub-module and a third determining sub-module. The second determining sub-module is configured to determine, based on the real-time positions of the plurality of unmanned systems and the real-time position of the obstacle, a first set of safety distances between the plurality of unmanned systems and the obstacle respectively, and a second set of safety distances between the plurality of unmanned systems and adjacent unmanned systems at the formation positions respectively; and the third determining sub-module is configured to determine, for each unmanned system, a comprehensive obstacle function value of the unmanned system by performing safety constraint on the follower unmanned system based on the first set of safety distances and the second set of safety distances.

[0177] According to an embodiment of the present application, the third determining sub-module comprises a first determining unit, a fusion unit and a second determining unit. The first determining unit is configured to determine a target set of safety distances of the unmanned system according to the first set of safety distances and the second set of safety distances; the fusion unit is configured to fuse safety constraint conditions in the target set of safety distances by using the comprehensive continuous obstacle function to obtain a safety index; and the second determining unit is configured to determine the comprehensive obstacle function value based on a smoothing parameter and the safety index.

[0178] According to an embodiment of the present application, the correction module 750 comprises a fourth determining sub-module, a fifth determining sub-module and a correction sub-module. The fourth determining sub-module is configured to determine a correction condition threshold value according to a dynamic variation of the comprehensive obstacle function value and a gain adjustment term, wherein the dynamic variation is obtained by processing a state vector of the follower unmanned system using a high-order derivative of the comprehensive obstacle function; the fifth determining sub-module is configured to determine that the predicted control parameter is the target control parameter in a case where the correction condition threshold value is greater than or equal to a preset correction threshold value; and the correction sub-module is configured to correct the predicted control parameter in a direction of a Lie derivative of the correction condition threshold value to obtain the target control parameter in a case where the correction condition threshold value is less than the preset correction threshold value.

[0179] According to an embodiment of the present application, the evaluation neural network comprises a formation controller, and the formation controller is configured to perform the following operations: for the i-th follower unmanned system, determining, based on the cost function, a cost value of the i-th candidate control parameter under the i-th formation error of the i-th follower unmanned system according to the i-th formation error of the i-th follower unmanned system and the i-th candidate control parameter; determining a real-time Bellman error based on a partial derivative of the cost value with respect to the i-th formation error and a j-th formation error variation value corresponding to the j-th candidate control parameter of the j-th follower unmanned system, wherein the j-th formation error variation value represents a change trend of the j-th formation error corresponding to the j-th candidate control parameter; and determining that the candidate control parameter is the reference control parameter in a case where the real-time Bellman error satisfies a first preset condition.

[0180] According to an embodiment of the present application, the evaluation neural network is trained based on the following manner: determining an experience Bellman error based on historical control parameters and historical formation errors of the unmanned system swarm; and updating adaptive weights of the evaluation neural network by using a gradient descent adaptive algorithm until a target Bellman error meets a second preset condition, wherein the target Bellman error is a sum of the experience Bellman error and a real-time Bellman error.

[0181] According to an embodiment of the present application, the device further comprises a switching module. The switching module is configured to switch the current formation to a target formation when it is detected that the obstacle distance between the plurality of obstacles is less than the current safety distance of the current formation of the unmanned system swarm, wherein the target safety distance of the target formation is less than the obstacle distance.

[0182] According to an embodiment of the present application, any of the first determining module 710, the first input module 720, the second input module 730, the second determining module 740 and the correction module 750 can be combined in one module, or any of them can be split into multiple modules. Alternatively, at least part of the function of one or more of these modules can be combined with at least part of the function of other modules, and implemented in one module. According to an embodiment of the present application, at least one of the first determining module 710, the first input module 720, the second input module 730, the second determining module 740 and the correction module 750 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging a circuit, etc. hardware or firmware, or in any one of software, hardware and firmware implementation or in any appropriate combination of any of them. Alternatively, at least one of the first determining module 710, the first input module 720, the second input module 730, the second determining module 740 and the correction module 750 can be at least partially implemented as a computer program module which can perform corresponding functions when it is run.

[0183] Figure 8 A block diagram of an electronic device suitable for implementing the collision avoidance learning control method applied to the unmanned system swarm according to an embodiment of the present application is shown.

[0184] As Figure 8As shown, the electronic device 800 according to an embodiment of the present application includes a processor 801 which can perform various appropriate actions and processes in accordance with a program stored in a read only memory (ROM) 802 or a program loaded from a storage section 808 into a random access memory (RAM) 803. The processor 801 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chip set, and / or a special purpose microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 801 can also include an on-board memory for cache use. The processor 801 can include a single processing unit or multiple processing units to perform the various actions of the method processes according to embodiments of the present application.

[0185] In the RAM 803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. The processor 801 performs various operations of the method processes according to embodiments of the present application by executing the programs in the ROM 802 and / or the RAM 803. Note that the programs can also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 can also perform various operations of the method processes according to embodiments of the present application by executing the programs stored in the one or more memories.

[0186] According to an embodiment of the present application, the electronic device 800 can further include an input / output (I / O) interface 805 which is also connected to the bus 804. The electronic device 800 can further include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as necessary. A removable recording medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 810 as necessary, so that a computer program read out therefrom is installed in the storage section 808 as necessary.

[0187] The application further provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or can exist independently without being assembled into the device / apparatus / system. The computer readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the application.

[0188] According to the embodiments of the application, the computer readable storage medium can be a non-volatile computer readable storage medium, which can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to the embodiments of the application, the computer readable storage medium can include the ROM 802 and / or the RAM 803 described above, and / or one or more memories other than the ROM 802 and the RAM 803.

[0189] The embodiments of the application also include a computer program product, which includes a computer program containing program codes for executing the method shown in the flow chart. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the method for collision avoidance learning control applied to a swarm of unmanned systems provided by the embodiments of the application.

[0190] The above functions defined in the system / apparatus of the embodiments of the application are performed when the computer program is executed by the processor 801. According to the embodiments of the application, the system, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0191] In one embodiment, the computer program can rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be transmitted, distributed, downloaded and installed in the form of a signal on a network medium, and be downloaded and installed through the communication part 809 and / or installed from the detachable medium 811. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to wireless, wired, etc., or any appropriate combination thereof.

[0192] In such embodiments, the computer program can be downloaded and installed from the network via the communication section 809, and / or installed from the removable media 811. When the computer program is executed by the processor 801, the above-described functions defined in the system of the embodiments of the present application are executed. The system, device, apparatus, module, unit, etc. described above can be realized by the computer program modules according to the embodiments of the present application.

[0193] According to the embodiments of the present application, the program code for executing the computer program provided by the embodiments of the present application can be written in any combination of one or more programming languages, and specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming language, and / or assembly / machine language. The programming language includes, but is not limited to, such as Java, C++, python, "C" language or similar programming language. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected to the Internet through an Internet service provider).

[0194] The flowcharts and block diagrams in the drawings illustrate the possible architectures, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a segment, or a portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or in the reverse order, depending on the functionality involved. It should also be noted that each block in the flowcharts or block diagrams, and combinations of blocks in the flowcharts or block diagrams, can be implemented by special-purpose hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0195] Those skilled in the art can understand that the features described in various embodiments of the present application can be combined and / or integrated in various combinations and / or integrations, even if such combinations or integrations are not explicitly described in the present application. In particular, the features described in various embodiments of the present application can be combined and / or integrated in various combinations and / or integrations without departing from the spirit and teachings of the present application. All such combinations and / or integrations fall within the scope of the present application.

[0196] The embodiments of the application have been described. However, these embodiments are merely for illustration and are not intended to limit the scope of the application. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Various alternatives and modifications to the embodiments described herein will be apparent to those skilled in the art in view of the foregoing without departing from the scope of the application.

Claims

1. A collision avoidance learning control method applied to a swarm of unmanned systems, characterized in that, The method comprises: determining, based on a dynamic model of the unmanned system, a state vector of each of a plurality of unmanned systems in a cluster of unmanned systems according to a real-time position of each of the plurality of unmanned systems, wherein the plurality of unmanned systems comprises a leading unmanned system and a plurality of follower unmanned systems, the dynamic model of the unmanned system is constructed based on a first continuous function, a second continuous function and a real-time control parameter of the unmanned system, the first continuous function represents a change in the state vector of the unmanned system under the influence of no control, and the second continuous function represents a mapping relationship between a change in the state vector of the unmanned system under the influence of control and a control parameter of the cluster of unmanned systems; obtaining a formation error of each of the plurality of unmanned systems in the cluster of unmanned systems according to a communication weight between the plurality of unmanned systems, the state vector of each of the unmanned systems and an expected formation vector input error dynamic model of each of the follower unmanned systems relative to the leading unmanned system, wherein the communication weight represents a degree of communication association between the plurality of unmanned systems, and the error dynamic model is constructed based on a control mapping parameter of the unmanned system corresponding to the first continuous function and a control-free change in the state vector of the unmanned system corresponding to the second continuous function; for each of the unmanned systems, inputting the formation error into an evaluation neural network to obtain a predicted control parameter; determining a comprehensive obstacle function value based on the real-time position of each of the plurality of unmanned systems and a real-time position of an obstacle, wherein the comprehensive obstacle function value is used to adjust the formation of the cluster of unmanned systems under the condition that a safety constraint condition is met; correcting the predicted control parameter based on the comprehensive obstacle function value to obtain a target control parameter, so as to control the cluster of unmanned systems to perform formation movement by using the target control parameter; wherein the evaluation neural network comprises a formation controller, and the formation controller is configured to perform the following operations: for an i-th follower unmanned system in the cluster of unmanned systems, determining, based on a cost function, a cost value of an i-th candidate control parameter under an i-th formation error of the i-th follower unmanned system and the i-th candidate control parameter, wherein i is greater than or equal to 1 and less than N, i is not equal to j, and N is a number of unmanned systems in a communication set that communicates with the i-th follower unmanned system. determining a real-time Bellman error based on a partial derivative of the cost value with respect to the i-th formation error and a j-th formation error change value corresponding to a j-th candidate control parameter of a j-th follower unmanned system, wherein the j-th formation error change value represents a change trend of the j-th formation error corresponding to the j-th candidate control parameter, i and j are greater than or equal to 1 and less than N, i is not equal to j, and N is a number of unmanned systems in a communication set that communicates with the i-th follower unmanned system. in a case where the real-time Bellman error meets a first preset condition, determining that the candidate control parameter is a reference control parameter, wherein the reference control parameter is used to evaluate a training result of the evaluation neural network.

2. The method of claim 1, wherein, The dynamic model of the unmanned system is constructed in the following manner: determine m-order states of the unmanned system according to a real-time position of the unmanned system, to compose the state vector of the unmanned system, where m is an integer greater than or equal to 1; construct the dynamic model of the unmanned system according to the state vector, the first continuous function, the second continuous function and the real-time control parameter of the unmanned system.

3. The method of claim 2, wherein, The error dynamic model is constructed according to a communication topology graph of the unmanned system cluster, where the communication topology graph includes nodes corresponding to the unmanned systems respectively, edges connecting the nodes, and the communication weights, and the edges represent communication relationships between the unmanned systems.

4. The method of claim 3, wherein, The error dynamic model includes error dynamic sub-models corresponding to the follower unmanned systems respectively, and the error dynamic sub-models are constructed based on the following manners: determine a gain coefficient based on a communication weight and a containment gain between the ith follower unmanned system and the jth follower unmanned system in the communication set; obtain a controlled change of a state vector of the jth follower unmanned system according to a control mapping parameter of the jth follower unmanned system corresponding to the second continuous function and a real-time control parameter of the jth follower unmanned system; obtain an intermediate change of the state vector of the jth follower unmanned system according to a sum of an uncontrolled change of the state vector of the jth follower unmanned system corresponding to the first continuous function and the controlled change; obtain a target change of the state vector of the jth follower unmanned system according to the intermediate change of the state vector and a change of an expected formation vector of the jth follower unmanned system relative to the leader unmanned system; construct an error dynamic sub-model of the ith follower unmanned system according to the gain coefficient and the target change.

5. The method of claim 1, wherein, The determination of the comprehensive obstacle function value based on the real-time positions of the unmanned systems and the real-time position of the obstacle includes: determine a first safety distance set between the unmanned systems and the obstacle respectively and a second safety distance set between the unmanned systems and adjacent unmanned systems in formation positions respectively based on the real-time positions of the unmanned systems and the real-time position of the obstacle; for each unmanned system, perform safety constraint on the follower unmanned system based on the first safety distance set and the second safety distance set, and determine the comprehensive obstacle function value of the unmanned system.

6. The method of claim 5, wherein, The safety constraint on the follower unmanned system based on the first safety distance set and the second safety distance set to determine the comprehensive obstacle function value of the unmanned system includes: determine a target safety distance set of the unmanned system according to the first safety distance set and the second safety distance set; fuse the safety constraint conditions in the target safety distance set by using a comprehensive continuous obstacle function to obtain a safety index; determine the comprehensive obstacle function value based on a smoothing parameter and the safety index.

7. The method of claim 1, wherein, The correction of the predicted control parameter based on the comprehensive obstacle function value to obtain a target control parameter includes: The correction condition threshold is determined according to a dynamic change amount of the comprehensive disorder function value and a gain adjustment term, wherein the dynamic change amount is obtained by processing a state vector of the following unmanned system using a high-order derivative of the comprehensive disorder function; In a case where the correction condition threshold is greater than or equal to a preset correction threshold, the prediction control parameter is determined as the target control parameter; In a case where the correction condition threshold is less than the preset correction threshold, the prediction control parameter is corrected in a Lie derivative direction of the correction condition threshold to obtain the target control parameter.

8. The method of claim 1, wherein, The evaluation neural network is trained based on the following manner: An experience Bellman error is determined based on historical control parameters and historical formation errors of the unmanned system cluster; An adaptive weight of the evaluation neural network is updated using a gradient descent adaptive algorithm until a target Bellman error satisfies a second preset condition, wherein the target Bellman error is a sum of the experience Bellman error and the real-time Bellman error.

9. The method of claim 1, wherein, The method further comprises: In a case where an obstacle distance between a plurality of the obstacles is less than a current safety distance of a current formation of the unmanned system cluster, the current formation is switched to a target formation, and a target safety distance of the target formation is less than the obstacle distance.

Images