Distributed decision-making method and device for multi-objective UAV swarm

Abstract

The present application relates to a distributed decision-making method and device for multiple targets of a drone cluster. The method includes: constructing a distributed decision-making finite state machine and a game decision neural network for drones in a drone cluster, adjusting the parameters of the game decision neural network in real time in the current decision round according to the decision data stored in the previous decision round and the real-time data currently acquired by the drone, setting the state jump rules in the distributed decision finite state machine, and the drones in the normal decision maker and the entangled state make decision outputs through the game decision neural network after real-time parameter adjustment, and the drones in the lost state determine the optimal task target of the lost as the decision output through the comparison relationship between the number of task targets and the number of drones within a preset distance, and the drones in the decision holder state maintain the decision of the previous decision round. The use of this method can cope with decision outputs in different scenarios.

Description

Technical Field

[0001] The present application relates to the technical field of autonomous decision-making of unmanned aerial vehicles, and in particular to a distributed decision-making method and device for multi-objective unmanned aerial vehicle clusters. Background Art

[0002] With the development of drone technology, the development of counter-drone technology has also been put on the agenda. This study uses drone multi-target task decision-making as the task scenario. When multiple drones perform tasks at the same time, the completion of drone cluster tasks can significantly improve the efficiency of task execution. However, in multi-task target scenarios, the efficiency of drone clusters in completing tasks will be significantly reduced. At present, one approach is to divide drone clusters into multiple drone sub-clusters through decision-making. However, the current conventional technology is that the control center communicates and interacts with the drone cluster, and executes the task by issuing decision instructions for the drone cluster. This method cannot be implemented after the communication interaction is lost. At present, there are also autonomous decision-making of drone clusters based on intelligent agents, but for a single drone, its decision is relatively simple and cannot cope with the complex situation of executing multiple mission targets at the same time. Summary of the invention

[0003] Based on this, it is necessary to provide a distributed decision-making method and device for multi-objective drone clusters that can autonomously switch drone states and flexibly output decisions according to different task difficulties in order to address the above technical problems.

[0004] A multi-objective distributed decision-making method for a drone swarm, the method comprising:

[0005] Constructing a distributed decision-making finite state machine and a game decision neural network for drones in a drone cluster; the states of the drones in the distributed decision-making finite state machine include: decision holder, normal decision maker, entangled person, and lost person;

[0006] In the current decision-making round, according to the decision data stored in the previous decision-making round stored by the drone and the real-time data currently acquired, real-time parameter adjustment is performed on the game decision neural network;

[0007] The state jump rule in the distributed decision finite state machine is set; the state jump rule includes: if the state of the distributed decision finite state machine of the drone is a normal decision maker, and it is judged that the distance between the drone and any task target is greater than the drone decision distance threshold, then the state of the drone is judged to jump to a decision holder; if it is judged that the short-term decision jump of the drone is greater than the short-term decision memory capacity of the drone multiplied by the short-term decision tolerance jump rate of the drone, then the state of the drone is judged to jump to a entangled person; if the distance of the drone that jumps to the decision holder from any task target is less than the decision distance threshold, the state of the decision holder drone jumps to a normal decision maker, if the current lost state bit of the drone in the decision holder is 1, the state of the decision holder drone jumps to a lost person, if the distance of the drone that jumps to the entangled person from any task target is greater than the decision distance threshold, the state of the entangled drone jumps to a decision holder, if the distance of the drone that jumps to the entangled person from all task targets increases or decreases, the state of the entangled drone jumps to a lost person, and if the number of steps counted by the drone that jumps to the lost person reaches the threshold, the state of the lost drone jumps to a normal decision maker;

[0008] The drones in the normal decision maker and the entangled state make decision outputs through the game decision neural network after real-time parameter adjustment. The drones in the lost state determine the optimal mission target of the lost as the decision output through the comparison relationship between the number of mission targets and the number of drones within a preset distance. The drones in the decision holder state maintain the decision of the previous decision round.

[0009] In one of the embodiments, it also includes: when the drone in the normal decision maker status satisfies the jump conditions of the entangled person and the decision keeper at the same time, if the normal decision maker satisfies the lost person condition after becoming the entangled person, it will jump to the entangled person first; if the normal decision maker does not satisfy the lost person condition after becoming the entangled person, it will jump to the decision keeper first.

[0010] In one of the embodiments, the drone is in a normal decision maker state and a entangled state. If the entangled state satisfies the jump conditions of the lost state and the decision keeper at the same time, it will enter the lost state first. If it does not meet the lost state conditions, it will become a decision keeper. If the normal decision maker meets the entry conditions of the decision keeper, it will directly become a decision keeper.

[0011] In one of the embodiments, it also includes: setting a decision hold flag, a tangle flag and a lost flag; when the state of the drone is in the decision holder, the decision hold flag is 1, otherwise it is 0; when the state of the drone is in the tangle, the tangle flag is 1, otherwise it is 0; when the state of the drone is in the lost, the lost flag is 1, otherwise it is 0.

[0012] In one of the embodiments, it also includes: if the number of drones within a preset distance of the drone of the lost person is equal to the number of mission targets, the drone of the lost person selects the mission target with the closest distance as the optimal mission target as the decision output; otherwise, the drone of the lost person selects the mission target with the smallest difference as the optimal mission target as the decision output.

[0013] In one embodiment, it further includes: assigning a decision variable weight to each decision variable, and constructing a game decision neural network according to the decision variable weight:

[0014]

[0015] Among them, W is the game decision neural network, represents the network coefficient, Decision variable weights According to the network coefficient Normalized to get, Represents network parameters;

[0016] In the current decision-making round, the decision data stored in the previous decision-making round stored by the drone and the currently acquired real-time data are input into the game decision neural network, so as to adjust the parameters of the game decision neural network in real time.

[0017] In one of the embodiments, it also includes: calculating a set of decision variable weights based on any two decision variables of the decision variables, and then expanding the calculated set of decision variable weights and another set of decision variable weights to obtain a third decision variable weight, thereby calculating all decision variable weights.

[0018] A multi-objective distributed decision-making device for a drone swarm, the device comprising:

[0019] A framework construction module is used to construct a distributed decision-making finite state machine and a game decision neural network for drones in a drone cluster; the states of the drones in the distributed decision-making finite state machine include: decision holder, normal decision maker, entangled person, and lost person;

[0020] A parameter adjustment module is used to adjust the parameters of the game decision neural network in real time in the current decision round according to the decision data stored in the previous decision round stored by the drone and the real-time data currently acquired;

[0021] The state jump logic setting module is used to set the state jump rules in the distributed decision-making finite state machine; the state jump rules include: if the state of the distributed decision-making finite state machine of the drone is a normal decision maker, and it is judged that the distance between the drone and any task target is greater than the drone decision distance threshold, then the state of the drone is judged to jump to a decision holder; if it is judged that the short-term decision jump of the drone is greater than the short-term decision memory capacity of the drone multiplied by the short-term decision tolerance jump rate of the drone, then the state of the drone is judged to jump to a tangle; if the drone that jumps to a decision holder is less than the decision distance threshold from any task target , the state of the decision holder's drone jumps to a normal decision maker. If the current lost state bit of the decision holder's drone is 1, the state of the decision holder's drone jumps to a lost one. If the distance of the drone that jumps to a tangled one from any mission target is greater than the decision distance threshold, the state of the tangled drone jumps to a decision holder. If the distance of the drone that jumps to a tangled one from all mission targets increases or decreases, the state of the tangled drone jumps to a lost one. If the number of steps counted by the drone that jumps to a lost one reaches the threshold, the state of the lost drone jumps to a normal decision maker;

[0022] The decision output module is used for the drones in the normal decision maker and the entangled state to output decisions through the game decision neural network after real-time parameter adjustment; the drones in the lost state determine the optimal mission target of the lost state as the decision output through the comparison relationship between the number of mission targets and the number of drones within a preset distance; and the drones in the decision holder state maintain the decision of the previous decision round.

[0023] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0024] Constructing a distributed decision-making finite state machine and a game decision neural network for drones in a drone cluster; the states of the drones in the distributed decision-making finite state machine include: decision holder, normal decision maker, entangled person, and lost person;

[0025] In the current decision-making round, according to the decision data stored in the previous decision-making round stored by the drone and the real-time data currently acquired, real-time parameter adjustment is performed on the game decision neural network;

[0026] The state jump rule in the distributed decision finite state machine is set; the state jump rule includes: if the state of the distributed decision finite state machine of the drone is a normal decision maker, and it is judged that the distance between the drone and any task target is greater than the drone decision distance threshold, then the state of the drone is judged to jump to a decision holder; if it is judged that the short-term decision jump of the drone is greater than the short-term decision memory capacity of the drone multiplied by the short-term decision tolerance jump rate of the drone, then the state of the drone is judged to jump to a entangled person; if the distance of the drone that jumps to the decision holder from any task target is less than the decision distance threshold, the state of the decision holder drone jumps to a normal decision maker, if the current lost state bit of the drone in the decision holder is 1, the state of the decision holder drone jumps to a lost person, if the distance of the drone that jumps to the entangled person from any task target is greater than the decision distance threshold, the state of the entangled drone jumps to a decision holder, if the distance of the drone that jumps to the entangled person from all task targets increases or decreases, the state of the entangled drone jumps to a lost person, and if the number of steps counted by the drone that jumps to the lost person reaches the threshold, the state of the lost drone jumps to a normal decision maker;

[0027] The drones in the normal decision maker and the entangled state make decision outputs through the game decision neural network after real-time parameter adjustment. The drones in the lost state determine the optimal mission target of the lost as the decision output through the comparison relationship between the number of mission targets and the number of drones within a preset distance. The drones in the decision holder state maintain the decision of the previous decision round.

[0028] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0029] Constructing a distributed decision-making finite state machine and a game decision neural network for drones in a drone cluster; the states of the drones in the distributed decision-making finite state machine include: decision holder, normal decision maker, entangled person, and lost person;

[0030] In the current decision-making round, according to the decision data stored in the previous decision-making round stored by the drone and the real-time data currently acquired, real-time parameter adjustment is performed on the game decision neural network;

[0031] The state jump rule in the distributed decision finite state machine is set; the state jump rule includes: if the state of the distributed decision finite state machine of the drone is a normal decision maker, and it is judged that the distance between the drone and any task target is greater than the drone decision distance threshold, then the state of the drone is judged to jump to a decision holder; if it is judged that the short-term decision jump of the drone is greater than the short-term decision memory capacity of the drone multiplied by the short-term decision tolerance jump rate of the drone, then the state of the drone is judged to jump to a entangled person; if the distance of the drone that jumps to the decision holder from any task target is less than the decision distance threshold, the state of the decision holder drone jumps to a normal decision maker, if the current lost state bit of the drone in the decision holder is 1, the state of the decision holder drone jumps to a lost person, if the distance of the drone that jumps to the entangled person from any task target is greater than the decision distance threshold, the state of the entangled drone jumps to a decision holder, if the distance of the drone that jumps to the entangled person from all task targets increases or decreases, the state of the entangled drone jumps to a lost person, and if the number of steps counted by the drone that jumps to the lost person reaches the threshold, the state of the lost drone jumps to a normal decision maker;

[0032] The drones in the normal decision maker and the entangled state make decision outputs through the game decision neural network after real-time parameter adjustment. The drones in the lost state determine the optimal mission target of the lost as the decision output through the comparison relationship between the number of mission targets and the number of drones within a preset distance. The drones in the decision holder state maintain the decision of the previous decision round.

[0033] The above-mentioned distributed decision-making method, device, computer equipment and storage medium for multi-objective of drone clusters are designed with distributed finite state machines according to different target decision scenarios that drones may face, and the states and jump conditions of the finite state machines are set. In addition, a distributed game decision neural network is constructed to achieve the optimal target selection when considering different decision variables under local information of drones. The network realizes the adaptive adjustment of the weight coefficients of each decision variable under local information through the historical data of environmental information of drones and real-time observations. That is, the drone can adjust parameters while flying. The present invention realizes the autonomous decision-making and selection of task targets by drones based on surrounding local information without communication, and can be flexibly applied to a variety of drone decision-making task target selection task scenarios, and has the practicality of real machine deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a flow chart of a multi-objective distributed decision-making method for a drone swarm in one embodiment;

[0035] Figure 2 A schematic diagram of setting state jump rules in a distributed decision-making finite state machine in one embodiment;

[0036] Figure 3 A schematic diagram of the structure of a game decision neural network in another embodiment;

[0037] Figure 4 1 is a simulation diagram of multiple dynamic motions of a swarm of different target threat drones countering a target in an embodiment; wherein, Figures (a)-(i) represent simulation diagrams of time steps t=0s, t=380s, t=700s, t=1000s, t=1200s, t=1500s, t=1930s, t=2263s, and t=2862s, respectively;

[0038] Figure 5 It is a simulation diagram of multiple dynamic motions of a swarm of threatening drones against the same target in an embodiment; wherein, Figures (a)-(i) represent simulation diagrams of time steps t=0 s, t=380 s, t=700 s, t=1000 s, t=1200 s, t=1500 s, t=1930 s, t=2263 s, and t=2862 s, respectively;

[0039] Figure 6 It is a structural block diagram of a distributed decision-making device for multi-objectives of a drone cluster in one embodiment;

[0040] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] In one embodiment, Figure 1 As shown, a distributed decision-making method for multi-objectives of a drone cluster is provided, comprising the following steps:

[0043] Step 102, constructing a distributed decision-making finite state machine and a game decision neural network for drones in the drone cluster.

[0044] The states of the drone in the distributed decision-making finite state machine include: decision holder, normal decision maker, entangled and lost.

[0045] Finite-state machine (FSM), also known as finite state automaton, is a mathematical model that represents a finite number of states and behaviors such as transitions and actions between these states.

[0046] Step 104, in the current decision round, the game decision neural network is adjusted in real time according to the decision data stored in the previous decision round by the drone and the real-time data currently obtained.

[0047] Step 106, setting state jump rules in the distributed decision-making finite state machine.

[0048] like Figure 2 As shown, the state jump rules include: if the state of the distributed decision finite state machine of the drone is a normal decision maker, and it is judged that the distance between the drone and any task target is greater than the decision distance threshold of the drone, then the state of the drone is judged to jump to a decision holder; if it is judged that the short-term decision jump of the drone is greater than the short-term decision memory capacity of the drone multiplied by the short-term decision tolerance jump rate of the drone, then the state of the drone is judged to jump to a entangled person; if the distance of the drone that jumps to the decision holder from any task target is less than the decision distance threshold, the state of the decision holder's drone jumps to a normal decision maker, if the current lost state bit of the drone in the decision holder is 1, the state of the decision holder's drone jumps to a lost person, if the distance of the drone that jumps to the entangled person from any task target is greater than the decision distance threshold, the state of the entangled drone jumps to a decision holder, if the distance of the drone that jumps to the entangled person from all task targets increases or decreases, the state of the entangled drone jumps to a lost person, and if the counted number of steps of the drone that jumps to the lost person reaches the threshold, the state of the lost drone jumps to a normal decision maker.

[0049] Step 108, the drones in the normal decision maker and entangled states make decision outputs through the game decision neural network after real-time parameter adjustment. The drones in the lost state determine the optimal mission target as the decision output through the comparison relationship between the number of mission targets and the number of drones within a preset distance. The drones in the decision holder state maintain the decision of the previous decision round.

[0050] In the above-mentioned distributed decision-making method for multi-objectives of drone swarms, a distributed finite state machine is designed according to the different target decision-making scenarios that the drone may face, and the state and jump conditions of the finite state machine are set. In addition, a distributed game decision neural network is constructed to achieve the optimal target selection when considering different decision variables under local information of the drone. The network realizes the adaptive adjustment of the weight coefficients of each decision variable under local information through the historical data of the drone's environmental information and real-time observations. That is, the drone adjusts parameters while flying. The present invention realizes the autonomous decision-making and selection of task targets by drones based on the surrounding local information in a non-communication environment, and can be flexibly applied to a variety of drone decision-making task target selection task scenarios, and has the practicality of real machine deployment.

[0051] In one embodiment, when a drone in a normal decision maker status satisfies the jump conditions of both the entangled and decision keeper status, if the normal decision maker satisfies the lost condition after becoming the entangled, the drone will be prioritized to jump to the entangled status; if the normal decision maker does not satisfy the lost condition after becoming the entangled, the drone will be prioritized to jump to the decision keeper status.

[0052] In another embodiment, the drone is in the normal decision maker and entangled states. If the entangled state satisfies the transition conditions of the lost state and the decision keeper at the same time, it will preferentially enter the lost state. If it does not meet the lost state conditions, it will become a decision keeper. If the normal decision maker meets the entry conditions of the decision keeper, it will directly become a decision keeper. The above embodiment shows that when the drone can meet multiple state transition conditions in one state, the drone will preferentially jump to certain states to complete the decision-making of various complex situations.

[0053] In one of the embodiments, a decision hold flag, a entangled flag and a lost flag are set; when the state of the drone is in the decision holder, the decision hold flag is 1, otherwise it is 0; when the state of the drone is in the entangled, the entangled flag is 1, otherwise it is 0; when the state of the drone is in the lost, the lost flag is 1, otherwise it is 0.

[0054] Specifically, a pseudo code for distributed decision-making of multi-objective UAV clusters is proposed. The algorithm is as follows:

[0055] Input: Decision variables (distribution of drones in the drone cluster , the distance between the drone and the mission target , target threat level , which can be expanded according to the actual task), and its corresponding initial weight coefficient ; UAV short-term decision memory capacity ; Maximum allowable distribution error ; Drone decision distance threshold .

[0056] Output: The drone’s final decision .

[0057] 1. According to the different threat levels of different targets Calculate the amount of drones in the drone cluster that each target deserves

[0058] 2. Initialize drone target selection

[0059] 3. If the distribution error of the drone cluster is greater than the maximum allowable distribution error

[0060] 4. if (The actual number of steps the drone moves is less than the data capacity required for the network to adaptively adjust the parameters)

[0061] 5. The drone stores the data required by the game decision neural network, and the game decision neural network does not make adaptive adjustments

[0062] 6. else

[0063] 7. The drone updates the data required for the game decision neural network

[0064] 8. The drone achieves adaptive adjustment of decision weights through game decision neural network.

[0065] 9. end if

[0066] 10. end if

[0067] 11. The drone records the decision value as a historical decision before updating the decision

[0068] 12. If the distance between the drone and any mission target is greater than the drone decision distance threshold

[0069] 13. The drone determines whether it is a decision holder and updates the decision holding flag

[0070] 14. end if

[0071] 15. UAV computational game neural network target priority sorting sequence

[0072] 16. If Drones are decision makers

[0073] 17. The drone keeps its recent historical decisions unchanged

[0074] 18. else

[0075] 19. The drone selects the target according to the game neural network target priority sorting sequence

[0076] 20. end if

[0077] 21. The drone updates the environmental observation and re-obtains the number of drones around each target

[0078] 22. if (The drone's short-term decision memory capacity is full)

[0079] 23. UAV calculation of short-term decision jump number

[0080] 24. if (The short-term decision jump of the drone is greater than the short-term decision memory capacity of the drone multiplied by the short-term decision tolerance jump rate of the drone)

[0081] 25. (The entangled person's sign position is 1)

[0082] 26. The hesitant person finds the target of his hesitancy based on his short-term decision-making records.

[0083] 27. end if

[0084] 28. end if

[0085] 29. for target tin The entangler traverses the entangled target list. If the list is not empty, the program will automatically skip

[0086] 30. The obsessive person calculates the distance between himself and the object of his obsession

[0087] 31. end for

[0088] 32. if (The drone entanglement flag is 1)

[0089] 33. If the person who is struggling is getting closer or farther away from all the goals of his struggle

[0090] 34. (Lost sign position 1 of the drone)

[0091] 35. UAV calculation: ideal UAV distribution number for each target minus actual UAV distribution number

[0092] 36. If the distribution difference of drones around the target that the drone is entangled with is equal

[0093] 37. The drone selects the mission target closest to itself as the optimal target choice

[0094] 38. else

[0095] 39. UAV selection The target with the smallest (possibly negative) difference between the distribution of drones and the expected value is the optimal target selection.

[0096] 40. end if

[0097] 41. Lost Person Selection is set to the best target selection

[0098] 42. end if

[0099] 43. end if

[0100] 44. The drone updates the environmental observation and re-obtains the number of drones around each target

[0101] 45. if (The drone belongs to the lost person and the number of steps taken by the lost person to strengthen his belief is less than the maximum number of steps taken by the lost person to strengthen his belief)

[0102] 46. (The number of steps of the lost person's firm belief increases by 1)

[0103] 47. (The decision of the lost person remains the best choice for the lost person)

[0104] 48. (The drone's entangled person mark position is 0; the lost person mark position is 0; the lost person's belief step counter is reset to zero; the lost person's optimal choice position is zero)

[0105] 49. end if

[0106] 50. Output the final decision of the drone

[0107] For the design of state jump rules in distributed decision-making finite state machines, the drone is a normal decision maker in a normal state. When the number of decision jumps in the drone's short-term decision memory reaches a certain threshold, that is, the number of short-term decision jumps exceeds the drone's short-term decision memory capacity multiplied by the drone's short-term decision tolerance jump rate, the drone will change from a normal decision maker to a entangled person. The target selection mechanism of the entangled person is the same as that of the normal decision maker, and both calculate the preferred target through the distributed game decision neural network. However, if the entangled person is getting farther and farther away from or closer to all its entangled targets, then the entangled person state will become the lost person state. The lost person state is a substate of the entangled person state. The lost person will calculate the preferred target according to the rules. If the actual expected difference of the distribution values ​​of the drones around the lost person's entangled target (the lost person still has the entangled target of the entangled person state) is equal, the lost person selects the target closest to him as the optimal target; if the actual expected difference of the distribution values ​​of the drones around the lost person's entangled target is not equal, then the lost person selects the drone with the smallest actual expected difference in distribution values ​​(possibly negative) as his preferred target. After becoming a lost drone, it will firmly believe in its preferred goal and make a fixed decision for several steps. When the number of fixed decision steps reaches a certain threshold, the drone will jump out of the lost state and become a normal decision maker. The decision holder state appears when the drone is far away from all targets and appears in the initial stage when the drone receives a countermeasure task. That is, when the drone is in the normal decision maker state and the entangled state, if the distance from all targets is greater than the decision distance threshold, the drone will become a decision holder. Even if the entangled state flag of the drone is 1 when it is in the decision holder state, it will give priority to the decision according to the decision holder mechanism, so the state machine does not reflect the drone's transition from the decision holder state to the entangled state. However, if the lost state flag of the decision holder is 1, the decision holder will become a lost one.

[0108] In one embodiment, a decision variable weight is assigned to each decision variable, and a game decision neural network is constructed according to the decision variable weight:

[0109]

[0110] Among them, W is the game decision neural network, Represents network coefficients and decision variable weights According to the network coefficient Normalized to get, Represents network parameters;

[0111] In the current decision-making round, the decision data stored in the previous decision-making round stored by the drone and the currently acquired real-time data are input into the game decision neural network, so as to adjust the parameters of the game decision neural network in real time.

[0112] Specifically, Figure 3 As shown in Figure 2, assuming that there are 4 targets in the drone's field of view and 3 decision variables, the weight coefficients of the decision variables are , , , respectively represent the decision weight of drone distribution, the decision weight of drone distance from the target, and the target threat value (inversely proportional to the threat level). After comprehensively considering these decision factors, the drone calculates a decision value for each target and solves the optimal decision target through a minimum value unit. , , is uncertain, and all are initialized to 1 in this technology (and will be adjusted automatically later). Because different decision variables have different orders of magnitude, the values ​​and orders of magnitude of each decision variable may change during the actual flight of the drone. In order to better meet the needs of actual applications, it is necessary to achieve adaptive adjustment of the network parameters. Among them, the drone distribution data is obtained in real time through vision, the drone obtains the distance from the target through sensors and stores several adjacent steps as calculation data, and the target threat value is obtained through situation assessment. The above three sets of data form a matrix of the same dimension, which is used for parameter adaptive adjustment. Assume that the comprehensive evaluation value is , the initial coefficient of the neural network weight is , :

[0113]

[0114] Based on the game theory, the weight coefficients of the two networks are minimized. , By optimizing, the optimal weight can be obtained, that is, , . Determine the objective function:

[0115]

[0116] According to the differential properties of the matrix, the objective function can be equivalently transformed into a linear equation system for optimizing the first-order derivative conditions:

[0117]

[0118] The optimized network weight coefficient Normalize and finally determine the comprehensive evaluation value based on game theory combination weighting :

[0119]

[0120] In one embodiment, the above method can simultaneously update two sets of weight ratios. When the decision variables of the drone have three sets, , , The three groups of weights are assigned. At this time, the drone first calculates the distance data from the drone to the target based on the drone distribution data and the distance data from the drone to the target. , , and then calculate based on the drone distribution data and target threat value data , , the decision weights of the UAV distributions are unified according to the common decision variables. If there are more decision variables in the target selection task, this method can be used to expand the calculation of other decision weights.

[0121]

[0122] The drone only relies on the local information within the sensor range to adjust parameters. For example, if the distance between target 2 and drone 3 exceeds the drone decision distance threshold, drone 3 cannot obtain valid information about target 2. The drone only relies on the target information within the decision distance range to adjust the network weight coefficient.

[0123] If the user is more biased towards a certain decision factor, the coefficient of the decision factor in the game decision neural network weight can be manually increased. For example, if the user is more biased towards the distribution of drones emerging from the drone cluster meeting the target threat level, the user can adjust the weight of the decision factor manually through data adaptive adjustment. Later expanded to ( ). At the group level, the distribution of drones will converge to the value that focuses on this decision variable, but the whole process is completed only by drones relying on local information to make decisions.

[0124] Based on the embodiments of the present invention. In order to test the effectiveness and robustness of this research technology, a cluster of 40 drones is set up to capture 4 flexible drone targets for countermeasure targets. The threat area is set so that neither our drones nor the target drones can be exposed in the threat area and need to bypass. Our drones need to make autonomous decisions based on the situation of the enemy targets, and the corresponding drone distribution effects for each target emerge at the group level. This research technology is deployed in a simulation environment, and the experimental effects of the above experimental scenarios are demonstrated. The experimental environment is MATLAB R2023b.

[0125] Figure 4 It shows that 40 drones are clustered to counter 4 dynamic targets in an environment with a complex dynamic threat zone. The threat level of the 4 targets is the same. At this time, our drone cluster should show a uniform distribution effect. Figure 4The study demonstrated that 40 drones clustered to counter four dynamic targets in an environment with a complex dynamic threat zone. The threat levels of the four targets were different, and the threat levels of the four targets were set to 1:2:3:4. The decision factors of the drones mentioned that the drones in this study prioritized countering the target drone targets with higher threat levels, that is, the ideal distribution of counter drones around the four targets with different threat levels was also different.

[0126] In the above experiment, our drone will turn on its signal light to the target drone’s initial calibration color. Figure 5 It can be seen from the figure that when our drone cluster faces four different targets from the enemy, it can achieve a uniform distribution effect by simply making autonomous decisions based on local information and selecting its own counterattack targets. Figure 4 It can be seen that with a total of 40 drones, our drones have a distribution of 10:10:10:10 at the group level. Figure 4 It can be seen that when the target threat value is 1:2:3:4, our drones attack the four targets based on local information, and a drone distribution that is strongly correlated with the target threat level emerges.

[0127] The above simulation experiments and mission objectives were analyzed under multiple simulation tasks. 40 drones and 4 enemy dynamic moving targets were set up. Six groups of experiments were conducted under different target threat ratios, and the distribution ratio of drones targeting the 4 targets was statistically analyzed. The statistical results are shown in Table 1.

[0128] Table 1. Distribution statistics of drones in multiple experiments

[0129]

[0130] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0131] In one embodiment, Figure 6As shown, a distributed decision-making device for multi-objectives of a drone cluster is provided, including: a framework construction module 602, a parameter adjustment module 604, a state jump logic setting module 606 and a decision output module 608, wherein:

[0132] The framework construction module 602 is used to construct a distributed decision-making finite state machine and a game decision neural network for drones in the drone cluster; the states of the drones in the distributed decision-making finite state machine include: decision holder, normal decision maker, entangled person and lost person;

[0133] The parameter adjustment module 604 is used to adjust the parameters of the game decision neural network in real time in the current decision round according to the decision data stored in the previous decision round stored by the drone and the real-time data currently acquired;

[0134] The state jump logic setting module 606 is used to set the state jump rules in the distributed decision-making finite state machine; the state jump rules include: if the state of the distributed decision-making finite state machine of the drone is a normal decision maker, and it is judged that the distance between the drone and any task target is greater than the drone decision distance threshold, then the state of the drone is judged to jump to a decision holder; if it is judged that the short-term decision jump of the drone is greater than the short-term decision memory capacity of the drone multiplied by the short-term decision tolerance jump rate of the drone, then the state of the drone is judged to jump to a tangle; if the drone that jumps to a decision holder is less than the decision distance from any task target Threshold, the state of the decision holder's drone jumps to a normal decision maker. If the current lost state bit of the decision holder's drone is 1, the state of the decision holder's drone jumps to a lost one. If the distance of the drone that jumps to a tangled one from any mission target is greater than the decision distance threshold, the state of the tangled one's drone jumps to a decision holder. If the distance of the drone that jumps to a tangled one from all mission targets increases or decreases, the state of the tangled one's drone jumps to a lost one. If the number of steps counted by the drone that jumps to a lost one reaches the threshold, the state of the lost one's drone jumps to a normal decision maker.

[0135] The decision output module 608 is used for the drones in the normal decision maker and the entangled state to output decisions through the game decision neural network after real-time parameter adjustment; the drones in the lost state determine the optimal mission target of the lost state as the decision output through the comparison relationship between the number of mission targets and the number of drones within a preset distance; and the drones in the decision holder state maintain the decision of the previous decision round.

[0136] In one embodiment, the state jump logic setting module 606 is also configured that when the drone in the normal decision maker satisfies the jump conditions of the entangled and decision keeper at the same time, if the normal decision maker satisfies the lost condition after becoming the entangled, the state jump to the entangled is given priority; if the normal decision maker does not satisfy the lost condition after becoming the entangled, the state jump to the decision keeper is given priority.

[0137] In one embodiment, the state jump logic setting module 606 also sets the state of the drone in the normal decision maker and the entangled state. If the entangled state satisfies the jump conditions of the lost state and the decision keeper at the same time, it will enter the lost state first. If the lost state conditions are not met, it will become a decision keeper. If the normal decision maker meets the entry conditions of the decision keeper, it will directly become a decision keeper.

[0138] In one embodiment, the state jump logic setting module 606 also sets a decision hold flag, a tangle flag and a lost flag; when the state of the drone is in the decision holder, the decision hold flag is 1, otherwise it is 0; when the state of the drone is in the tangle, the tangle flag is 1, otherwise it is 0; when the state of the drone is in the lost, the lost flag is 1, otherwise it is 0.

[0139] In one embodiment, the decision output module 608 is also used to select the mission target with the closest distance as the optimal mission target as the decision output if the number of drones within a preset distance of the drone of the lost person is equal to the number of mission targets, otherwise the drone of the lost person selects the mission target with the smallest difference as the optimal mission target as the decision output.

[0140] In one embodiment, the parameter adjustment module 604 is further used to assign a decision variable weight to each decision variable, and construct a game decision neural network according to the decision variable weight as follows:

[0141]

[0142] Among them, W is the game decision neural network, Represents network coefficients and decision variable weights According to the network coefficient Normalized to get, Represents network parameters;

[0143] In the current decision-making round, the decision data stored in the previous decision-making round stored by the drone and the currently acquired real-time data are input into the game decision neural network, so as to adjust the parameters of the game decision neural network in real time.

[0144] In one embodiment, the decision output module 608 is also used to calculate a set of decision variable weights based on any two decision variables of the decision variables, and then expand the calculated set of decision variable weights and another set of decision variable weights to obtain a third decision variable weight, thereby calculating all decision variable weights.

[0145] For the specific definition of the distributed decision-making device for multi-objectives of drone swarms, please refer to the definition of the distributed decision-making method for multi-objectives of drone swarms above, which will not be repeated here. Each module in the above-mentioned distributed decision-making device for multi-objectives of drone swarms can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0146] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a distributed decision-making method for multiple objectives of a drone cluster is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse, etc.

[0147] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0148] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0149] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0150] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0151] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A distributed decision-making method for multi-objectives of drone swarms, characterized in that: The method comprises: Constructing a distributed decision-making finite state machine and a game decision neural network for drones in a drone cluster; the states of the drones in the distributed decision-making finite state machine include: decision holder, normal decision maker, entangled person, and lost person; In the current decision-making round, according to the decision data stored in the previous decision-making round stored by the drone and the real-time data currently acquired, real-time parameter adjustment is performed on the game decision neural network; The state jump rule in the distributed decision finite state machine is set; the state jump rule includes: if the state of the distributed decision finite state machine of the drone is a normal decision maker, and it is judged that the distance between the drone and any task target is greater than the drone decision distance threshold, then the state of the drone is judged to jump to a decision holder; if it is judged that the short-term decision jump of the drone is greater than the short-term decision memory capacity of the drone multiplied by the short-term decision tolerance jump rate of the drone, then the state of the drone is judged to jump to a entangled person; if the distance of the drone that jumps to the decision holder from any task target is less than the decision distance threshold, the state of the decision holder drone jumps to a normal decision maker, if the current lost state bit of the drone in the decision holder is 1, the state of the decision holder drone jumps to a lost person, if the distance of the drone that jumps to the entangled person from any task target is greater than the decision distance threshold, the state of the entangled drone jumps to a decision holder, if the distance of the drone that jumps to the entangled person from all task targets increases or decreases, the state of the entangled drone jumps to a lost person, and if the number of steps counted by the drone that jumps to the lost person reaches the threshold, the state of the lost drone jumps to a normal decision maker; The drones in the normal decision maker and entangled states make decision outputs through the game decision neural network after real-time parameter adjustment. The drones in the lost state determine the optimal mission target of the lost as the decision output through the comparison relationship between the number of mission targets and the number of drones within a preset distance. The drones in the decision holder state maintain the decision of the previous decision round.

2. The method according to claim 1, characterized in that The method further comprises: When a drone in the normal decision-maker position satisfies the jump conditions of both the entangled and decision-holder positions, if the normal decision-maker becomes a entangled person and then satisfies the lost person condition, the entangled person will be jumped first; If a normal decision maker becomes a tangle and does not meet the lost condition, the decision holder will be prioritized.

3. The method according to claim 1, characterized in that The method further comprises: The drone is in the normal decision maker and entangled states. If the entangled state meets the jump conditions of the lost state and the decision keeper at the same time, it will enter the lost state first. If it does not meet the lost state conditions, it will become a decision keeper. If the normal decision maker meets the entry conditions of the decision keeper, it will directly become a decision keeper.

4. The method according to claim 1, characterized in that: The method further comprises: Set the decision-keeping flag, the entangled flag, and the lost flag; When the state of the drone is in the decision holder, the decision holding flag is 1, otherwise it is 0; When the drone is in the state of a tangle, the tangle flag is 1, otherwise it is 0; When the drone is in the lost state, the lost flag is 1, otherwise it is 0.

5. The method according to claim 1, characterized in that The drone at the lost person determines the optimal mission target of the lost person as a decision output by comparing the number of mission targets and the number of drones within a preset distance, including: If the number of drones within the preset distance of the drone of the lost person is equal to the number of mission targets, the drone of the lost person selects the mission target with the closest distance as the optimal mission target as the decision output; otherwise, the drone of the lost person selects the mission target with the smallest difference as the optimal mission target as the decision output.

6. The method according to any one of claims 1 to 5, characterized in that: In the current decision-making round, according to the decision data stored in the previous decision-making round stored by the drone and the currently acquired real-time data, the game decision neural network is adjusted in real time, including: Assign a decision variable weight to each decision variable, and construct a game decision neural network based on the decision variable weight: ; in, is the game decision neural network, Represents network coefficients and decision variable weights According to the network coefficient Normalized to get, Represents network parameters; In the current decision-making round, the decision data stored in the previous decision-making round stored by the drone and the currently acquired real-time data are input into the game decision neural network, so as to adjust the parameters of the game decision neural network in real time.

7. The method according to claim 6, characterized in that The real-time parameter adjustment of the game decision neural network includes: A set of decision variable weights is calculated based on any two decision variables of the decision variables, and then the calculated set of decision variable weights and another set of decision variable weights are expanded to obtain a third decision variable weight, and all decision variable weights are calculated in this way.

8. A distributed decision-making device for multi-objective drone swarms, characterized in that: The device comprises: A framework construction module is used to construct a distributed decision-making finite state machine and a game decision neural network for drones in a drone cluster; the states of the drones in the distributed decision-making finite state machine include: decision holder, normal decision maker, entangled person, and lost person; A parameter adjustment module is used to adjust the parameters of the game decision neural network in real time in the current decision round according to the decision data stored in the previous decision round stored by the drone and the real-time data currently acquired; The state jump logic setting module is used to set the state jump rules in the distributed decision-making finite state machine; the state jump rules include: if the state of the distributed decision-making finite state machine of the drone is a normal decision maker, and it is judged that the distance between the drone and any task target is greater than the drone decision distance threshold, then the state of the drone is judged to jump to a decision holder; if it is judged that the short-term decision jump of the drone is greater than the short-term decision memory capacity of the drone multiplied by the short-term decision tolerance jump rate of the drone, then the state of the drone is judged to jump to a tangle; if the drone that jumps to a decision holder is less than the decision distance threshold from any task target , the state of the decision holder's drone jumps to a normal decision maker. If the current lost state bit of the decision holder's drone is 1, the state of the decision holder's drone jumps to a lost one. If the distance of the drone that jumps to a tangled one from any mission target is greater than the decision distance threshold, the state of the tangled drone jumps to a decision holder. If the distance of the drone that jumps to a tangled one from all mission targets increases or decreases, the state of the tangled drone jumps to a lost one. If the number of steps counted by the drone that jumps to a lost one reaches the threshold, the state of the lost drone jumps to a normal decision maker; The decision output module is used for the drones in the normal decision maker and entangled states to output decisions through the game decision neural network after real-time parameter adjustment; the drones in the lost state determine the optimal mission target of the lost state as the decision output through the comparison relationship between the number of mission targets and the number of drones within a preset distance; and the drones in the decision holder state maintain the decision of the previous decision round.

Images