Unmanned aerial vehicle competition decision-making method, device and equipment and storage medium

Abstract

The invention discloses an unmanned aerial vehicle competition decision-making method, device and equipment and a storage medium. The method comprises the following steps: constructing a competition situation evaluation model, a response region and an acquisition region of an unmanned aerial vehicle; based on the competitive situation evaluation model, the response area and the acquisition area, constructing an individual comprehensive reward model of the unmanned aerial vehicle; determining an unmanned aerial vehicle utility matrix based on the individual comprehensive reward model; based on a preset maneuvering strategy, a preset response strategy and the unmanned aerial vehicle utility matrix, constructing an unmanned aerial vehicle maneuvering-response decision dynamic game model; based on an improved grey wolf optimization algorithm, solving the maneuvering-response decision dynamic game model to obtain a maneuvering-response decision dynamic optimization result; and determining a target maneuvering-response strategy of the unmanned aerial vehicle based on a maneuvering-response decision dynamic optimization result.

Description

Technical Field

[0001] The present invention relates to the field of drone competition decision-making, and in particular to a drone competition decision-making method, device, equipment and storage medium. Background Art

[0002] Unmanned aerial vehicles (UAVs) are playing an increasingly important role in the modern information technology landscape. Due to payload limitations and the limited number of onboard devices, precise response timing is crucial to maximize device effectiveness. One-on-one competition, the foundation of swarm competition, is crucial for improving autonomous competitive capabilities by studying its coordinated maneuver-response decision-making.

[0003] Existing methods for drone maneuver and response decision-making primarily include game theory, reinforcement learning, and heuristic optimization, but all have limitations. Game theory methods, such as matrix games, have achieved initial success in one-on-one competitions, but their computational complexity is high. Especially in high-dimensional strategy spaces, linear programming methods for solving Nash equilibria are computationally expensive, making them difficult to meet the demands of real-time decision-making. Reinforcement learning methods have garnered attention in recent years in the field of drone decision-making. They learn optimal strategies through competitive training, but these methods rely on large amounts of data, have limited generalization capabilities in dynamic, uncertain environments, and are difficult to ensure strategy convergence. Furthermore, existing methods often treat maneuver and response decisions as independent problems, ignoring the mutual influence between the two. This results in a lack of consistency in overall competitive decision-making, impacting competitive effectiveness.

[0004] To improve optimization solution efficiency, heuristic optimization methods such as the Gray Wolf Optimization (GWO) algorithm, a swarm intelligence optimization algorithm based on biomimetic principles, simulate the hunting behavior of a gray wolf population to explore an optimization path toward a target within a multidimensional space and under various constraints. This algorithm overcomes the shortcomings of traditional algorithms such as the Particle Swarm Optimization (PSO) and Genetic Algorithm (GA), which often tend to quickly fall into local optima and experience slow iterative convergence. By drawing on the social organization and hunting patterns of gray wolves, it possesses a strong exploration capability. Improvements to existing algorithms can be made in three areas: population initialization, optimizing convergence factors, and enhancing local optimization capabilities. This allows the algorithm to achieve faster, more flexible, and more efficient intelligent search. However, due to the more complex gray wolf population optimization and position update methods, convergence time is long, and computational efficiency still fails to meet the real-time requirements of competitive competition. In highly dynamic and uncertain competitive environments, decision-making strategies must not only be highly effective but also possess efficient computational capabilities to ensure that competitive decisions can be made quickly.

[0005] Therefore, existing methods still face challenges such as high computational complexity, insufficient strategy coordination, and limited real-time performance when dealing with UAV maneuver-response collaborative decision-making problems. Summary of the Invention

[0006] In order to solve the above-mentioned technical problems existing in the prior art, the present invention provides a drone competition decision-making method, device, equipment and storage medium.

[0007] In order to achieve the above-mentioned purpose, the technical solution of the embodiment of the present invention is:

[0008] In a first aspect, an embodiment of the present invention provides a drone competition decision-making method, the method comprising:

[0009] Construct a UAV competitive situation assessment model, response area and acquisition area; the competitive situation assessment model includes: angle advantage function Speed ​​advantage function Distance advantage function and the height advantage function The competitive situation assessment model F role for: Wherein, the angle advantage function is expressed as: Where θ role and are the sight angles of the UAV and the target respectively; the speed advantage function is expressed as: Where, is the ideal speed of the UAV, expressed as: v max is the maximum scalar speed of the UAV, is the target speed, D max is the maximum response distance of the drone, v role =||v role ||, d rb =||P r -P b ||, d rb is the distance between red and blue, P r and P b are the position coordinates of the red and blue squares in three-dimensional space respectively; the distance advantage function is expressed as: Where D min To maintain a safe minimum distance between drones, D best is the optimal response distance of the UAV, δ is a positive real number; the height advantage function is expressed as: Where h role is the flight altitude of the UAV, h best is the optimal response height, h max is the maximum response height; hmin is the minimum response height; the response area is expressed as: The acquisition area is represented as:

[0010] Constructing an individual comprehensive reward model for the drone based on the competitive situation assessment model, the response area, and the acquisition area;

[0011] Determining a drone utility matrix based on the individual comprehensive reward model;

[0012] Based on the preset maneuver strategy, the preset response strategy and the drone utility matrix, a dynamic game model of drone maneuver-response decision-making is constructed;

[0013] Based on the improved grey wolf optimization algorithm, the maneuver-response decision dynamic game model is solved to obtain the maneuver-response decision dynamic optimization result;

[0014] Based on the dynamic optimization result of the maneuver-response decision, a target maneuver-response strategy of the UAV is determined.

[0015] In a second aspect, an embodiment of the present invention provides an unmanned competition decision-making device, the device comprising:

[0016] Construction module, used to construct the UAV's competitive situation assessment model, response area and acquisition area; the competitive situation assessment model includes: angle advantage function Speed ​​advantage function Distance advantage function and the height advantage function The competitive situation assessment model F role for: Wherein, the angle advantage function is expressed as: Where θ role and are the sight angles of the UAV and the target respectively; the speed advantage function is expressed as: Where, is the ideal speed of the UAV, expressed as: v max is the maximum scalar speed of the UAV, is the target speed, D max is the maximum response distance of the drone, v role =||v role ||, d rb =||P r -P b ||, d rb is the distance between red and blue, P r and P bare the position coordinates of the red and blue squares in three-dimensional space respectively; the distance advantage function is expressed as: Where D min To maintain a safe minimum distance between drones, D best is the optimal response distance of the UAV, δ is a positive real number; the height advantage function is expressed as: Where h role is the flight altitude of the UAV, h best is the optimal response height, h max is the maximum response height; h min is the minimum response height; the response area is expressed as: c fire ={d rb ≤D max andθ role ≤π / 2}; the acquisition area is expressed as:

[0017] The construction module is further configured to construct an individual comprehensive reward model for the drone based on the competitive situation assessment model, the response area, and the acquisition area;

[0018] a determination module, configured to determine a drone utility matrix based on the individual comprehensive reward model;

[0019] The construction module is further used to construct a UAV maneuver-response decision-making dynamic game model based on a preset maneuver strategy, a preset response strategy and the UAV utility matrix;

[0020] A solution module, configured to solve the maneuver-response decision dynamic game model based on an improved grey wolf optimization algorithm to obtain a maneuver-response decision dynamic optimization result;

[0021] The determination module is further configured to determine a target maneuver-response strategy of the UAV based on the maneuver-response decision dynamic optimization result.

[0022] In some embodiments, the individual comprehensive reward model includes: value function Cost function Reward and Penalty Function Response penalty function and the penalty function obtained Among them, the role variable role∈{r,b};

[0023] The drone individual comprehensive reward model for:

[0024] Among them, the value function of step k is defined as: Where, represents the damage level of the UAV at step k, represents the value function of the drone at step k-1, Indicates the acquisition conditions of the target, represents the number of response units emitted by the target at step k;

[0025] The cost function is defined as: Where Vm role is the drone response unit value, is the number of response units launched by the UAV at step k;

[0026] The reward function is defined as: Where, represents the value function of the target at step k, represents the competitive situation assessment model of the UAV at the kth step;

[0027] The response penalty function is defined as: Where Pf is the waste penalty value of the response unit;

[0028] The obtained penalty function is defined as: Where Pc is the penalty value obtained, Indicates that our drone has entered the target acquisition area.

[0029] In some embodiments, the determination module is further configured to determine a drone utility function based on the individual comprehensive reward model; the drone utility function is: Where, is the comprehensive reward model for the red team. is the blue team's individual comprehensive reward model; based on the utility function, the drone utility matrix is ​​determined; the drone utility matrix includes: the red team's utility matrix MR(k) and the blue team's utility matrix MB(k):

[0030]

[0031] In some embodiments, the improved gray wolf optimization algorithm is determined by: randomly initializing the population number and the number of iterations of each gray wolf parameter in the initial gray wolf optimization algorithm to obtain the initialized gray wolf optimization algorithm; based on the initialized gray wolf optimization algorithm, the introduced preset superior strategies are repeatedly eliminated with extremely weak inferior strategies to obtain a refined population strategy set, and the historical optimal fitness value of each population is output; based on the historical optimal fitness value, it is determined whether the preset end condition is met; if the preset end condition is met, the mixed strategy Nash equilibrium solution is directly output; if the preset end condition is not met, the above steps are repeated until the preset end condition is met.

[0032] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory for storing executable instructions; and a processor for implementing the above-mentioned unmanned competition decision-making method when executing the executable instructions stored in the memory.

[0033] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one executable instruction, and the at least one executable instruction is loaded and executed by a processor to implement the above-mentioned unmanned competition decision-making method.

[0034] The unmanned competitive decision-making method provided by the present invention first constructs a competitive situation assessment model, a response area, and an acquisition area of ​​a UAV; based on the competitive situation assessment model, the response area, and the acquisition area, an individual comprehensive reward model of the UAV is constructed; based on the individual comprehensive reward model, a utility matrix of the UAV is determined; secondly, based on a preset maneuvering strategy, a preset response strategy, and the UAV utility matrix, a dynamic game model of a UAV maneuver-response decision is constructed; based on an improved gray wolf optimization algorithm, the dynamic game model of a maneuver-response decision is solved to obtain a dynamic optimization result of a maneuver-response decision; finally, based on the dynamic optimization result of a maneuver-response decision, a target maneuver-response strategy of the UAV is determined. In this way, the dynamic maneuver-response decision-making game model proposed in the present invention can simultaneously consider the dynamic decision-making process of maneuver and response and their coupling relationship, enhance the integrity of the decision-making chain, and is more suitable for aerial competition scenarios; in addition, the improved gray wolf optimization algorithm can significantly improve the calculation speed based on the gray wolf optimization algorithm, so that it can meet the real-time needs of competition; on the other hand, the decision-making model and algorithm design of the present invention can effectively cope with complex dynamic arena environments, ensure that drones make the best decisions under rapidly changing competition conditions, and improve competition efficiency; at the same time, the present invention not only provides decision-making support under the framework of game theory in theory, but also verifies its effectiveness through empirical research, providing a reliable theoretical basis and practical guidance for drone competition decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic diagram of the structure of the drone competition decision-making system provided by an embodiment of the present invention;

[0036] Figure 2 This is a flow chart of a drone competition decision-making method provided by an embodiment of the present invention;

[0037] Figure 3 1 is a flow chart of the improved grey wolf optimization algorithm provided by an embodiment of the present invention for solving the update of the UAV maneuver-response strategy;

[0038] Figure 4 This is a schematic diagram of the structure of a drone competition decision-making device provided by an embodiment of the present invention;

[0039] Figure 5 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0041] In the following description, references to "some embodiments" describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present invention have the same meaning as commonly understood by those skilled in the art to which the embodiments of the present invention pertain. The terms used in the embodiments of the present invention are for the purpose of describing the embodiments of the present invention only and are not intended to limit the present invention.

[0042] The following describes an exemplary application of the unmanned competition decision-making device of an embodiment of the present invention. The unmanned competition decision-making device provided by the embodiment of the present invention can be implemented as a terminal or as a server. In one implementation, the unmanned competition decision-making device provided by the embodiment of the present invention can be implemented as various types of terminals such as laptop computers, tablet computers, desktop computers, mobile devices, etc.; in another implementation, the unmanned competition decision-making device provided by the embodiment of the present invention can also be implemented as a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present invention. The following describes an exemplary application of the unmanned competition decision-making device when it is implemented as a server.

[0043] See also Figure 1 , Figure 1: is a structural diagram of the unmanned competition decision-making system 10 provided in an embodiment of the present invention. In order to realize the control of the drone cluster, the embodiment of the present invention can provide an unmanned competition decision-making platform, which can be implemented as an unmanned competition decision-making application. The unmanned competition decision-making system 10 provided in an embodiment of the present invention includes a terminal 110, a network 120 and a server 130, wherein the server 130 is a server of the unmanned competition decision-making application. The server 130 can constitute the unmanned competition decision-making device of the embodiment of the present invention. The terminal 110 is connected to the server 130 through the network 120. The network 120 can be a wide area network or a local area network, or a combination of the two.

[0044] In some embodiments, please refer to Figure 1 When collaboratively deciding on a drone's competitive maneuver response, terminal 110 transmits the obtained drone competitive decision task to server 130 via network 120. In response to the drone competitive decision task initiated by terminal 110, server 130 constructs a competitive situation assessment model, response area, and acquisition area for the drone. Based on the competitive situation assessment model, response area, and acquisition area, server 130 constructs an individual comprehensive reward model for the drone. Based on the individual comprehensive reward model, server 130 determines the drone's utility matrix. Based on the preset maneuver strategy, preset response strategy, and drone utility matrix, server 130 constructs a dynamic game model for drone maneuver-response decision making. Using the improved gray wolf optimization algorithm, server 130 solves the dynamic game model for drone maneuver-response decision making to obtain a dynamic optimization result for drone maneuver-response decision making. Based on the dynamic optimization result for drone maneuver-response decision making, server 130 determines the drone's target maneuver-response strategy. After obtaining the drone's target maneuver-response strategy, server 130 transmits the target maneuver-response strategy to terminal 110 via network 120.

[0045] The embodiment of the present invention provides a method for making decisions in unmanned competition. Figure 2 , Figure 2 This is a flowchart of an unmanned competition decision-making method provided by an embodiment of the present invention, which will be combined with Figure 2 The steps shown are explained.

[0046] Step S210: constructing a competition situation assessment model, a response area, and an acquisition area for the UAV.

[0047] In the present invention, the competitive situation assessment model includes: angle advantage function Speed ​​advantage function Distance advantage function and the height advantage function The competitive situation assessment model F role for:

[0048] Among them, the angle advantage function is expressed as: Where θ role and are the sight angles of the UAV and the target, respectively.

[0049] The speed advantage function is expressed as: Where, is the ideal speed of the UAV, expressed as: v max is the maximum scalar speed of the UAV, is the target speed, D max is the maximum response distance of the drone, v role =||v role ||, d rb =||P r -P b ||, d rb is the distance between red and blue, P r and P b are the position coordinates of the red and blue squares in three-dimensional space respectively.

[0050] The distance advantage function is expressed as: Where D min To maintain a safe minimum distance between drones, D best is the optimal response distance of the UAV, and δ is a positive real number.

[0051] The height advantage function is expressed as: Where h role is the flight altitude of the UAV, h best is the optimal response height, h max is the maximum response height; h min is the minimum response height.

[0052] The response area is expressed as: c fire ={d rb ≤D max andθ role ≤π / 2}; the obtained area is expressed as:

[0053] In some embodiments, the competitive situation assessment model is a model used to evaluate the situation and status of the drone in the competitive environment. It can comprehensively consider multiple factors, such as the drone's own position, speed, altitude, posture, the target drone's position, action status, and arena environmental factors (such as terrain, weather conditions, etc.). It uses certain algorithms and logic to judge the pros and cons of the current competitive situation and provide a basis for subsequent collaborative decision-making.

[0054] In some embodiments, the response zone refers to the spatial range within which a drone can effectively deliver an attack to a target during a competition. In practice, the determination of this zone depends on the performance of the drone's response unit system, such as range, accuracy, and angle of attack. Within the response zone, a drone's response unit has a higher probability of hitting and damaging the target.

[0055] In some embodiments, an acquisition zone refers to the area within which a drone, under certain conditions, can successfully lock onto, track, and potentially acquire a target. Acquisition can take various forms, such as using sensors to lock onto a target drone and bringing it within the effective range of its response unit system. This zone is related to the drone's sensor capabilities, target characteristics, and other factors.

[0056] In the present invention, by collecting and analyzing various competition-related information, a competition situation assessment model is established to quantify the current competition situation; the response area and acquisition area are determined to clarify the response and control range of the drone in the competition.

[0057] Step S220: constructing an individual comprehensive reward model for the drone based on the competition situation assessment model, the response area, and the acquisition area.

[0058] In some embodiments, based on the competitive situation assessment model, response area, and acquisition area information, a model can be established for each drone to measure the rewards or benefits of its various behaviors during the competition. This model is used to measure the comprehensive rewards generated by the various behaviors and decisions of individual drones during the competition. Rewards can include benefits such as hitting the target, entering a favorable competitive position, occupying a favorable position, and successfully avoiding the target response. These factors can be quantified to guide the collaborative decision-making of drones.

[0059] In this invention, we use the competitive situation assessment results obtained in the previous step, along with the drone's response and control range information, to build a model to evaluate the rewards associated with various drone behaviors during competition. This reward model comprehensively considers multiple factors related to the drone, providing a quantitative basis for subsequent decision-making.

[0060] Step S230: Determine the drone utility matrix based on the individual comprehensive reward model.

[0061] In some embodiments, a drone utility matrix is ​​a matrix determined based on an individual comprehensive reward model and used to describe the utility values ​​of a drone under different decisions or states. The elements in the matrix reflect the benefits or value gained by the drone taking different actions in specific situations, providing data support for subsequent decision-making models.

[0062] Step S240: constructing a UAV maneuver-response decision-making dynamic game model based on the preset maneuver strategy, the preset response strategy and the UAV utility matrix.

[0063] In some embodiments, a preset maneuvering strategy refers to a pre-set set of strategies for a drone to perform maneuvering flight in a competition, such as a combination of climbing, diving, turning, accelerating, decelerating, and other actions, as well as maneuvering selection rules under different competitive situations.

[0064] In some embodiments, a preset response strategy refers to a pre-set strategy for the drone to respond using the response unit system, including strategies on when to respond, which response unit to select, and which target to respond to.

[0065] In some embodiments, a dynamic game model for drone maneuver-response decision-making combines drone maneuver and response strategies, taking into account the interactions and competition between multiple drones. In this model, each drone must choose its optimal maneuver and response strategy based on the possible actions of other drones to maximize its own interests.

[0066] In this paper, we combine pre-defined drone maneuver and response strategies with a drone utility matrix, viewing competition as a dynamic game process and constructing a model to describe the drone's decision-making regarding maneuver and response. This model considers the interaction between the drone and the target and the decision-making process, analyzing the optimal decision through game theory.

[0067] Step S250 , solving the maneuver-response decision dynamic game model based on the improved grey wolf optimization algorithm to obtain a maneuver-response decision dynamic optimization result.

[0068] In the present invention, an improved Gray Wolf Optimization Algorithm can be determined by presetting a superior strategy and an initial Gray Wolf Optimization Algorithm. Here, the pre-set superior strategy refers to a strategy pre-set in game theory. This strategy is superior to other strategies in certain situations. That is, regardless of the strategies adopted by other participants, the participant adopting the superior strategy will always achieve a better result or at least not be worse than other strategies.

[0069] In this paper, the original Gray Wolf optimization algorithm is improved using a pre-defined superior strategy. By incorporating the advantages of the superior strategy and adjusting the algorithm's parameters and search mechanism, an improved Gray Wolf optimization algorithm is obtained. The purpose of the improvement is to improve the algorithm's performance and efficiency in solving competitive decision-making problems.

[0070] It should be noted that the existing initial Gray Wolf Optimization algorithm suffers from slow convergence when faced with game models with large strategy spaces. To address this issue, the present invention introduces a mechanism for repeatedly eliminating extremely weakly inferior strategies. During each iteration of the initial Gray Wolf Optimization algorithm, extremely weakly inferior strategies—those whose fitness values ​​are surpassed by extremely weakly superior strategies—are eliminated, resulting in an improved Gray Wolf Optimization algorithm. This algorithm refines the strategy space of the game model, further improving the computational speed of the initial Gray Wolf Optimization algorithm while ensuring that the correctness of the approximate Nash equilibrium strategy is not affected.

[0071] Step S260: Determine the target maneuver-response strategy of the UAV based on the maneuver-response decision dynamic optimization result.

[0072] In some embodiments, the target maneuver-response strategy of a UAV refers to a decision plan for the specific maneuvers and responses used by the UAV in competition, determined based on the dynamic optimization results of the maneuver-response decision, that is, the specific decision results such as when and how the UAV performs maneuver flight, and when and how to use the response unit to respond.

[0073] The unmanned competitive decision-making method provided by the present invention first constructs a competitive situation assessment model, a response area, and an acquisition area of ​​a UAV; based on the competitive situation assessment model, the response area, and the acquisition area, an individual comprehensive reward model of the UAV is constructed; based on the individual comprehensive reward model, a utility matrix of the UAV is determined; secondly, based on a preset maneuvering strategy, a preset response strategy, and the UAV utility matrix, a dynamic game model of a UAV maneuver-response decision is constructed; based on an improved gray wolf optimization algorithm, the dynamic game model of a maneuver-response decision is solved to obtain a dynamic optimization result of a maneuver-response decision; finally, based on the dynamic optimization result of a maneuver-response decision, a target maneuver-response strategy of the UAV is determined. In this way, the dynamic maneuver-response decision-making game model proposed in the present invention can simultaneously consider the dynamic decision-making process of maneuver and response and their coupling relationship, enhance the integrity of the decision-making chain, and is more suitable for general competitive scenarios; in addition, the improved gray wolf optimization algorithm can significantly improve the calculation speed based on the gray wolf optimization algorithm, so that it can meet the real-time needs of competition; on the other hand, the decision-making model and algorithm design of the present invention can effectively cope with complex dynamic arena environments, ensure that drones make the best decisions under rapidly changing competitive conditions, and improve competitive efficiency; at the same time, the present invention not only provides decision-making support under the framework of game theory in theory, but also verifies its effectiveness through empirical research, providing a reliable theoretical basis and practical guidance for drone competition decision-making.

[0074] In some embodiments, the individual comprehensive reward model includes: value function Cost function Reward and Penalty Function Response penalty function and the penalty function obtained Among them, the role variable role∈{r,b}; Note that in the following formula, as long as the form is or The variables are all related variables of the target of the drone corresponding to the role.

[0075] Drone individual comprehensive reward model for:

[0076] Among them, the value function of step k is defined as: Where, represents the damage level of the UAV at step k, represents the value function of the drone at step k-1, Indicates the acquisition conditions of the target, represents the number of response units emitted by the target at step k.

[0077] The cost function is defined as: Where Vm role is the response unit value of the drone, is the number of response units launched by the UAV at step k.

[0078] The reward function is defined as: Where, represents the value function of the target at step k, Represents the competitive situation assessment model of the UAV at the kth step.

[0079] The response penalty function is defined as: Where Pf is the waste penalty value of the response unit.

[0080] The penalty function is defined as: Where Pc is the penalty value obtained, Indicates that our drone has entered the target acquisition area.

[0081] In some embodiments, the above step S230 can be implemented through steps S231 to S232:

[0082] Step S231: Determine the drone utility function based on the individual comprehensive reward model.

[0083] Here, the drone utility function is: Where, is the comprehensive reward model for the red team. It is the comprehensive reward model for the blue team individuals.

[0084] Step S232: Determine the drone utility matrix based on the utility function.

[0085] Here, the drone utility matrix includes: the red party utility matrix MR(k) and the blue party utility matrix MB(k):

[0086]

[0087] In some embodiments, the improved grey wolf optimization algorithm is determined by:

[0088] Step S261, randomly initialize the population size and iteration number of each gray wolf parameter in the initial gray wolf optimization algorithm to obtain an initialized gray wolf optimization algorithm.

[0089] Step S262: Based on the initialized grey wolf optimization algorithm, the extremely weak inferior strategies are repeatedly eliminated from the introduced preset superior strategies to obtain a refined population strategy set, and the historical optimal fitness value of each population is output.

[0090] Step S263: Based on the historical optimal fitness value, determine whether a preset end condition is met.

[0091] Step S264: If the preset end condition is met, directly output the mixed strategy Nash equilibrium solution.

[0092] Step S265: If the preset end condition is not met, repeat the above steps until the preset end condition is met.

[0093] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0094] This example proposes a dynamic maneuver-fire decision game model (DMFD) based on dynamic game theory to enhance the linkage between strategies and improve the consistency of competitive decision-making. Furthermore, this model is solved using an improved Grey Wolf optimization algorithm to improve computational efficiency and meet the decision-making requirements of real-time competitive games.

[0095] In order to achieve the above object, the present invention adopts the following technical solutions: Figure 3 As shown:

[0096] 1.1 Definition of the Relative Advantage Function (i.e., the Competitive Situation Assessment Model in the Above-mentioned Embodiment)

[0097] In a one-on-one drone competition, the main participants are the red drone r and the blue drone b.

[0098] (1) Angle advantage function

[0099] During the competition, the angle between the drone's velocity vector and the line of sight will affect the competition situation. For the pursuing drone, the smaller the angle, the better the response effect. Therefore, the angle advantage function of the drone in competition is defined as:

[0100]

[0101] Where θ role and are the sight angles of the UAV and the target, respectively.

[0102] If the velocities of the red square r and the blue square b are v r and v b , then:

[0103]

[0104] (2) The speed advantage function is:

[0105]

[0106] Where, is the ideal speed of the drone, v max is the maximum scalar speed of the UAV, is the target speed, D max is the maximum response distance of the drone, d rb is the distance between red and blue, P r and P b are the position coordinates of the red and blue squares in three-dimensional space respectively.

[0107] (3) The distance advantage function is:

[0108]

[0109] Where D min To maintain a safe minimum distance between drones, D best is the optimal response distance of the UAV, and δ is a positive real number.

[0110] (4) The height advantage function is:

[0111]

[0112] Where h role is the flight altitude of the UAV, h best is the optimal response height, h max is the maximum response height; h min is the minimum response height.

[0113] In summary, the relative advantage function of the UAV relative to the target is:

[0114]

[0115] And w1+w2+w3+w4=1.

[0116] 1.2 Individual Welfare Function (i.e., the Individual Comprehensive Reward Model in the Above-mentioned Embodiment)

[0117] During the competition, define the drone's response area: c fire ={d rb ≤D max andθ role ≤π / 2}; Get the area:

[0118] (1) Value function: Each UCAV has an initial value at the beginning of the competition, which gradually decreases during the competition due to the opponent's response. In previous studies, the hit probability of the response unit is usually set as a constant. However, the actual hit probability of each time is affected by the arena situation, that is, the comprehensive advantage. Therefore, the value function of the kth step is defined as follows:

[0119]

[0120] Where, represents the damage level of the UAV at step k, represents the value function of the drone at step k-1, Indicates the acquisition conditions of the target, represents the number of response units emitted by the target at step k.

[0121] (2) Cost function: Each UAV has a limited number of response units, so each launch will incur a corresponding cost. This part of the cost function is defined as:

[0122]

[0123] Where Vm role is the drone response unit value, is the number of response units launched by the UAV at step k.

[0124] (3) Reward function: During the attack, using the response unit can bring benefits to the UCAV, but the value of the benefit depends on whether the target is within the acquisition area. Therefore, the reward function is defined as follows:

[0125]

[0126] Where, represents the value function of the target at step k, Represents the competitive situation assessment model of the UAV at the kth step.

[0127] (4) Response penalty function: In order to prevent the UCAV from not responding when the acquisition conditions are met, or responding when the response conditions are not met, the penalty function is defined as follows:

[0128]

[0129] Where Pf is the penalty for wasting response units, which is designed to prevent the UCAV from using response units when the response conditions are not met. In addition, this penalty mechanism will incentivize the UCAV to use response units when the adversary enters its acquisition area.

[0130] (5) Acquisition penalty function: To prevent the drone from being acquired by the target, the acquisition penalty function is defined as follows:

[0131]

[0132] Where Pc is the penalty value obtained, Indicates that the drone has entered the acquisition zone of the target.

[0133] In summary, the individual welfare function of the drone at step k is defined as:

[0134]

[0135] 1.3 Dynamic Maneuver-Response Decision Game Model

[0136] The three elements of a game model are participants, a strategy set, and a payoff function. In a one-on-one drone competition, the main participants are drone r (red) and drone b (blue). The strategy set primarily consists of a maneuver strategy and a response strategy, while the utility function (payoff function) is primarily related to the individual welfare functions of both parties, as follows:

[0137] (1) Participant set: N = {r, b};

[0138] (2) Policy Set: in, represents the drone’s response strategy, Then it represents the maneuvering strategy of the drone role∈N. Among them, the maneuvering strategy set and response strategy set of the drone are shown in Table 1 below:

[0139] Table 1 UAV maneuver and response action library

[0140]

[0141] As shown in the table above, the drone has 15 maneuver strategies and 2 response strategies, for a total of 30 combined maneuver-response strategies. In other words, each drone’s strategy space consists of 30 strategies.

[0142] (3) Payoff function: When the strategy combination of the red drone and the blue drone is The payoff functions of the red and blue parties are:

[0143]

[0144] From this, the payment matrix of red and blue parties is as follows:

[0145]

[0146] Among them, each element in MR(k) represents the payoff when the red team adopts the rowth pure strategy against the colth pure strategy of the blue team. Correspondingly, each element in MB(k) can also be explained.

[0147] 1.4 Improved Gray Wolf Optimization Algorithm

[0148] The initial Grey Wolf Optimization Algorithm is optimized and improved, and a new improved Grey Wolf Optimization Algorithm is proposed to solve the dynamic optimization result of the maneuver-response decision of the above game model, namely the mixed strategy Nash equilibrium, thereby guiding the maneuver-response decision of both parties.

[0149] In addition, in the DMFD game model, each step requires updating the utility matrix, which further increases the computational burden. To simplify the calculation of the mixed strategy Nash equilibrium, the superior strategy in the mixed strategy is introduced.

[0150] The superior strategy in mixed strategy: Given two mixed strategies of player p If the strategy σ p Strictly dominates strategy if:

[0151]

[0152] Strategy σ p Weakly Dominates Strategy if

[0153]

[0154] Strategy σ p Very weakly dominates strategy if

[0155]

[0156] If formula (19) holds true, then the strategy is called σ p Strong superiority; if formula (20) holds, then the strategy is called σ p Weak superiority; if formula (21) holds, then the strategy is called σ p Extremely weak and superior.

[0157] In game theory, players are assumed to be completely rational and will choose the optimal strategy. Outperformed strategies, whether strictly outperformed, weakly outperformed, or very weakly outperformed, are eliminated. The strategy space is optimized by iteratively eliminating outperformed strategies, ensuring that the remaining strategy combinations contain all mixed strategy Nash equilibria. Eliminating weakly outperformed or very weakly outperformed strategies can simplify the game scale. This method uses an iterative elimination method for very weakly outperformed strategies, as shown in Algorithm 1:

[0158]

[0159] In Algorithm 1, GetMinColMat(MR) returns a Boolean matrix in which the minimum element of each column of matrix MR is set to 1 and the other elements are 0. GetMinColMat(MR) also returns a Boolean matrix in which the minimum element of each row of matrix AAA is set to 1 and the other elements are 0.

[0160] It is worth noting that the improved Grey Wolf Optimization Algorithm optimizes the strategy space of matrix games based on the concept of superior strategies. Therefore, the algorithm is applicable to the equilibrium calculation of general matrix games with a finite strategy space and is not limited to the DMFD model proposed in this paper.

[0161] In the present invention, the operation steps of the improved gray wolf optimization algorithm are as follows:

[0162] Step 1: Initialization

[0163] 1) After starting the algorithm, you must first initialize the population size parameters and the number of iterations. You need to enter the size (i.e., the number of populations, representing the number of strategies available to both sides) and the dimension (the characteristic dimensions of the strategies) of the red and blue strategy spaces. Also, set the maximum number of iterations (the maximum number of times the algorithm can run). These parameters provide the foundation for subsequent iterative optimization of the population size.

[0164] 2) Randomly initialize the position and velocity of each population.

[0165] Step 2: Strategy Refinement

[0166] According to the pseudocode in Algorithm 1, the population's strategy set is refined by repeatedly eliminating extremely weak strategies. By eliminating these strategies, the strategy space is narrowed, computational effort is reduced, and the population is more focused on potential optimal strategies. After this process is completed, the population's historical best fitness value, the population's historical best fitness value, and the historical best fitness value of all populations are output. These values ​​reflect the population's optimal performance during the strategy refinement process.

[0167] Step 3: Loop judgment

[0168] Determine whether the termination condition is met. The termination condition is generally when the preset maximum number of iterations is reached or the change in fitness value is less than a certain threshold. If the termination condition is met, the optimal solution is directly output and the algorithm ends; if not, the algorithm proceeds to the subsequent iterative update step.

[0169] Step 4: Population status update

[0170] Speed ​​and position update: Update the position of each population to simulate the two parties in the game constantly adjusting their own strategies in the strategy space.

[0171] Second strategy refinement: Repeatedly eliminate extremely weak strategies according to the pseudocode in Algorithm 1, further refine the strategy set, and continuously optimize the strategy space.

[0172] Fitness value calculation: Calculate the fitness value of each population to measure the pros and cons of each strategy combination (i.e., the position of the population).

[0173] Individual Optimal Update: Update the historical optimal fitness value and position of each individual in the population. If the current population fitness value is better than its historical optimal fitness value, then update the historical optimal fitness value and corresponding position.

[0174] Global Optimal Update: Update the historical optimal fitness value and position of the entire population. By comparing the fitness values ​​of all populations, determine the current global optimal strategy combination and its corresponding optimal fitness value.

[0175] Step 5: Update other parameters

[0176] Update other related parameters, such as inertia weight, number of iterations, etc. The inertia weight is usually adjusted dynamically as the number of iterations increases to balance the global and local search capabilities of the algorithm; the number of iterations is increased by 1 to record the progress of the algorithm.

[0177] Step 6: End output

[0178] When the termination condition is met, the optimal solution is output, that is, the mixed strategy Nash equilibrium solution of the game model, and the algorithm ends.

[0179] Figure 4Schematic diagram of the structure of the unmanned competition decision-making device provided by an embodiment of the present invention. Figure 4 As shown, the unmanned competition decision-making device 400 includes: a construction module 401 for constructing a competition situation assessment model, a response area and an acquisition area of ​​the drone; the competition situation assessment model includes: an angle advantage function Speed ​​advantage function Distance advantage function and the height advantage function The competitive situation assessment model F role for: Wherein, the angle advantage function is expressed as: Where θ role and are the sight angles of the UAV and the target respectively; the speed advantage function is expressed as: Where, is the ideal speed of the UAV, expressed as: v max is the maximum scalar speed of the UAV, is the target speed, D max is the maximum response distance of the drone, v role =||v role ||, d rb =||P r -P b ||, d rb is the distance between red and blue, P r and P b are the position coordinates of the red and blue squares in three-dimensional space respectively; the distance advantage function is expressed as: Where D min To maintain a safe minimum distance between drones, D best is the optimal response distance of the UAV, δ is a positive real number; the height advantage function is expressed as: Where h role is the flight altitude of the UAV, h best is the optimal response height, h max is the maximum response height; h min is the minimum response height; the response area is expressed as: The acquisition area is represented as: The construction module 401 is also used to construct an individual comprehensive reward model of the UAV based on the competitive situation assessment model, the response area and the acquisition area; the determination module 402 is used to determine the UAV utility matrix based on the individual comprehensive reward model; the construction module 401 is also used to construct a UAV maneuver-response decision dynamic game model based on a preset maneuver strategy, a preset response strategy and the UAV utility matrix; the solution module 403 is used to solve the maneuver-response decision dynamic game model based on the improved gray wolf optimization algorithm to obtain a maneuver-response decision dynamic optimization result; the determination module 402 is also used to determine the target maneuver-response strategy of the UAV based on the maneuver-response decision dynamic optimization result.

[0180] It should be noted that the description of the device embodiment of the present invention is similar to the description of the above-mentioned method embodiment, and has similar beneficial effects as the same method embodiment, so it will not be repeated. For technical details not disclosed in the device embodiment, please refer to the description of the method embodiment of the present invention for understanding.

[0181] It should be noted that, in the embodiment of the present invention, if the above-mentioned unmanned competitive decision-making method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a terminal to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read Only Memory), a magnetic disk or an optical disk. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

[0182] Based on the above embodiments, an embodiment of the present invention further provides an electronic device, Figure 5 FIG. 1 is a structural diagram of an electronic device according to an embodiment of the present invention. Figure 5 As shown, the hardware entity of the electronic device 500 includes: a memory 501 and a processor 502, wherein the memory 501 stores a computer program that can be run on the processor 502, and when the processor 502 executes the program, the steps in the drone competition decision-making method in the above embodiment are implemented.

[0183] The memory 501 is configured to store instructions and applications executable by the processor 502, and can also cache data to be processed or processed by the processor 502 and various modules in the electronic device 500 (for example, image data, audio data, voice communication data and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0184] Based on the foregoing embodiments, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor of an electronic device, it can implement the drone competition decision-making method provided in any of the foregoing embodiments.

[0185] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0186] The methods disclosed in the various method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0187] The features disclosed in the various product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0188] The features disclosed in the various method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0189] An embodiment of the present invention provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to execute the method provided by the embodiment of the present invention, for example, Figure 2 The method shown.

[0190] In some embodiments, the above-mentioned storage medium can be a computer-readable storage medium, such as ferroelectric random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disk, or compact disk read-only memory (CD-ROM) and other memories; it can also be various devices including one or any combination of the above-mentioned memories.

[0191] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0192] As examples, executable instructions may, but need not necessarily, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as one or more scripts in a Hypertext Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions). As examples, executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected by a communication network.

[0193] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present invention are included in the scope of protection of the present invention.

[0194] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are for description only and do not represent the advantages and disadvantages of the embodiments.

[0195] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, an element defined by the statement "comprises a..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed.

[0196] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A drone competition decision-making method, characterized in that: The method comprises: Construct a UAV competitive situation assessment model, response area and acquisition area; the competitive situation assessment model includes: angle advantage function Speed ​​advantage function Distance advantage function and the height advantage function The competitive situation assessment model F role for: Wherein, the angle advantage function is expressed as: Where θ role and are the sight angles of the UAV and the target respectively; the speed advantage function is expressed as: Where, is the ideal speed of the UAV, expressed as: v max is the maximum scalar speed of the UAV, is the target speed, D max is the maximum response distance of the drone, v role =||v role ||, d rb =||P r -P b ||, d rb is the distance between red and blue, P r and P b are the position coordinates of the red and blue squares in three-dimensional space respectively; the distance advantage function is expressed as: Where D min To maintain a safe minimum distance between drones, D best is the optimal response distance of the UAV, δ is a positive real number; the height advantage function is expressed as: Where h role is the flight altitude of the UAV, h best is the optimal response height, h max is the maximum response height; h min is the minimum response height; the response area is expressed as: c fire ={d rb ≤D ma an x d θ role ≤π / 2}; the acquisition area is expressed as: Constructing an individual comprehensive reward model for the drone based on the competitive situation assessment model, the response area, and the acquisition area; Determining a drone utility matrix based on the individual comprehensive reward model; Based on the preset maneuver strategy, the preset response strategy and the drone utility matrix, a dynamic game model of drone maneuver-response decision-making is constructed; Based on the improved grey wolf optimization algorithm, the maneuver-response decision dynamic game model is solved to obtain the maneuver-response decision dynamic optimization result; Based on the dynamic optimization result of the maneuver-response decision, a target maneuver-response strategy of the UAV is determined.

2. The method according to claim 1, characterized in that The individual comprehensive reward model includes: value function Cost function Reward and Penalty Function Response penalty function and the penalty function obtained Among them, the role variable role∈{r,b}; The drone individual comprehensive reward model for: Among them, the value function of step k is defined as: Where, represents the damage level of the UAV at step k, represents the value function of the drone at step k-1, Indicates the acquisition conditions of the target, represents the number of response units emitted by the target at step k; The cost function is defined as: Where Vm role is the drone response unit value, is the number of response units launched by the UAV at step k; The reward function is defined as: Where, represents the value function of the target at step k, represents the competitive situation assessment model of the UAV at the kth step; The response penalty function is defined as: Where Pf is the waste penalty value of the response unit; The obtained penalty function is defined as: Where Pc is the penalty value obtained, Indicates that our drone has entered the target acquisition area.

3. The method according to claim 2, characterized in that Determining the drone utility matrix based on the individual comprehensive reward model includes: Based on the individual comprehensive reward model, the drone utility function is determined; the drone utility function is: Where, is the comprehensive reward model for the red team. It is the comprehensive reward model for the blue team individuals; Based on the utility function, the drone utility matrix is ​​determined; the drone utility matrix includes: the red side utility matrix MR(k) and the blue side utility matrix MB(k):

4. The method according to claim 1, wherein The improved grey wolf optimization algorithm is determined by the following method: Randomly initializing the population size and the number of iterations of each gray wolf parameter in the initial gray wolf optimization algorithm to obtain an initialized gray wolf optimization algorithm; Based on the initialized grey wolf optimization algorithm, the extremely weak inferior strategies are repeatedly eliminated from the introduced preset superior strategies to obtain a refined population strategy set, and the historical optimal fitness value of each population is output; Based on the historical optimal fitness value, determining whether a preset end condition is met; If the preset end condition is met, the mixed strategy Nash equilibrium solution is directly output; If the preset end condition is not met, the above steps are repeated until the preset end condition is met.

5. An unmanned competitive decision-making device, characterized in that: The device comprises: Construction module, used to construct the UAV's competitive situation assessment model, response area and acquisition area; the competitive situation assessment model includes: angle advantage function Speed ​​advantage function Distance advantage function and the height advantage function The competitive situation assessment model F role for: Wherein, the angle advantage function is expressed as: Where θ role and are the sight angles of the UAV and the target respectively; the speed advantage function is expressed as: Where, is the ideal speed of the UAV, expressed as: v max is the maximum scalar speed of the UAV, is the target speed, D max is the maximum response distance of the drone, v role =||v role ||, d rb =||P r -P b ||, d rb is the distance between red and blue, P r and P b are the position coordinates of the red and blue squares in three-dimensional space respectively; the distance advantage function is expressed as: Where D min To maintain a safe minimum distance between drones, D best is the optimal response distance of the UAV, δ is a positive real number; the height advantage function is expressed as: Where h role is the flight altitude of the UAV, h best is the optimal response height, h max is the maximum response height; h min is the minimum response height; the response area is expressed as: c fire ={d rb ≤D max andθ role ≤π / 2}; the acquisition area is expressed as: The construction module is further configured to construct an individual comprehensive reward model for the drone based on the competitive situation assessment model, the response area, and the acquisition area; a determination module, configured to determine a drone utility matrix based on the individual comprehensive reward model; The construction module is further used to construct a UAV maneuver-response decision-making dynamic game model based on a preset maneuver strategy, a preset response strategy and the UAV utility matrix; A solution module, configured to solve the maneuver-response decision dynamic game model based on an improved grey wolf optimization algorithm to obtain a maneuver-response decision dynamic optimization result; The determination module is further configured to determine a target maneuver-response strategy of the UAV based on the maneuver-response decision dynamic optimization result.

6. An electronic device, characterized in that: include: a memory for storing executable instructions; The processor is configured to implement the unmanned competition decision-making method according to any one of claims 1 to 5 when executing the executable instructions stored in the memory.

7. A computer-readable storage medium, wherein at least one executable instruction is stored in the computer-readable storage medium, and the at least one executable instruction is loaded and executed by a processor to implement the unmanned competition decision-making method according to any one of claims 1 to 5.

Images