An unmanned aerial vehicle target tracking method, system and device based on a stochastic potential game

Abstract

The application discloses a kind of unmanned vehicle target tracking method, system and equipment based on random potential game, it is related to unmanned vehicle target tracking field, the method includes: obtaining the initial state of unmanned vehicle and target in simulation environment, and state parameter is constructed based on initial state;State parameter is input into unmanned vehicle navigator and target navigator respectively, and initial state is updated;Cost function is calculated based on updated state;Based on cost function, the update gradient of unmanned vehicle navigator and target navigator is calculated using deep reinforcement learning framework;Unmanned vehicle navigator and target navigator are trained based on update gradient;Updated unmanned vehicle navigator is deployed to unmanned vehicle, and the target state and unmanned vehicle state obtained in actual scene are used to optimize trained unmanned vehicle navigator;The tracking of target is realized based on optimized unmanned vehicle navigator.The application can improve the robustness of unmanned vehicle tracking capability.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) target tracking technology, and in particular to a UAV target tracking method, system, and device based on stochastic potential game theory. Background Technology

[0002] With the rapid advancement of intelligent distributed control algorithms, multi-UAV systems (hereinafter referred to as multi-UAV systems) are playing a crucial role in both military and civilian fields. The UAVs mentioned include any controllable unmanned powered aircraft such as rotorcraft, fixed-wing aircraft, and airships. In military applications, multi-UAV systems can serve as independent mission units or auxiliary units for manned aircraft, performing reconnaissance, surveillance, and strike missions. In civilian applications, deploying multi-UAV systems in disaster relief operations or infrastructure inspections not only saves manpower and material resources but also improves mission efficiency. In these applications, multi-UAV systems must simultaneously possess environmental adaptation, target tracking, and multi-angle observation capabilities to effectively deal with non-cooperative targets. Currently, multi-UAV systems have initially acquired intelligent characteristics, and tracking algorithms based on fused sensor data are relatively mature. However, the following problems still exist in practical application scenarios:

[0003] (1) Environmental uncertainty exists. Restrictions such as obstacles, dangers, or no-fly zones cause the environment to change dynamically when multi-aircraft systems perform target tracking tasks, and the information obtained by multi-aircraft systems through sensors is subject to noise or interference, which reduces the reliability and completeness of the status information;

[0004] (2) The targets being tracked are mostly non-cooperative entities. When multi-machine systems perform tasks such as intelligence, surveillance and reconnaissance, the targets being tracked may be non-cooperative entities. The targets may take measures such as evasion and interference. Most target tracking methods do not consider the game relationship between the system and the target. Currently, tracking algorithms based on game relationships mainly focus on zero-sum game scenarios and do not consider the impact of unknown environments on game relationships. Summary of the Invention

[0005] The purpose of this invention is to provide a UAV target tracking method, system, and device based on stochastic potential game theory, which considers the impact of environmental uncertainty and the non-cooperative behavior of the tracked object on the task, enabling multi-UAV systems to be practical and improving the robustness of tracking capabilities.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A method for UAV target tracking based on stochastic potential game theory includes:

[0008] The initial states of the UAV and the target in the simulation environment are obtained, and state parameters are constructed based on the initial states. The state parameters include: the relative positions of the UAV and the target, the relative positions of multiple UAVs, the observation values ​​of the UAVs, and the observation values ​​of the target.

[0009] The initial state is updated based on the state parameters in the UAV navigator and the target navigator.

[0010] The cost function is calculated based on the updated state; the form of the cost function satisfies the conditions of a stochastic potential game.

[0011] Based on the cost function, the update gradients of the UAV navigator and the target navigator are calculated using a deep reinforcement learning framework;

[0012] The UAV navigator and the target navigator are trained based on the updated gradient;

[0013] The updated drone navigator is deployed to the drone, and the trained drone navigator is optimized by obtaining the target state and drone state in the real scene.

[0014] Target tracking is achieved using an optimized drone navigator.

[0015] Optionally, based on the state parameters, the initial state is updated according to the data in the UAV navigator and the target navigator, specifically including:

[0016] The state parameters are input into the UAV navigator to obtain the expected values ​​of the UAV at the current moment; the expected values ​​include the expected heading and the expected speed.

[0017] Based on the expected value, the control quantity is calculated using an airborne flight control algorithm;

[0018] Based on the control variables, the UAV dynamics model is used to calculate the state of the UAV at the next moment;

[0019] The status parameters are input into the target navigator to obtain navigation instructions;

[0020] Based on the navigation instructions, the target's state at the next moment is calculated using a target dynamics model.

[0021] Optionally, the cost function is calculated using the following formula:

[0022]

[0023] Among them, R i Let represent the cost function of the i-th agent, which includes a drone and a target. Let i represent the tracking cost function of the i-th agent. Let i represent the enclosing cost function of the i-th agent. Let represent the action smoothing cost function of the i-th agent. Let represent the collision cost function of the i-th agent.

[0024] Optionally, the tracking cost function The calculation formula is as follows:

[0025]

[0026] Where, β trac Indicates the tracking ratio coefficient. v represents the target's maximum ground speed. max This indicates the maximum ground speed of the drone. Let be the rate of change of the relative distance between drone j and target k, and N represent the total number of drones.

[0027] Optionally, the enclosing cost function The calculation formula is as follows:

[0028]

[0029] Where, β sur This represents the enclosing ratio factor. This represents the angle between the line connecting the i-th drone and the target and the line connecting the (i+1)-th drone and the target.

[0030] Optionally, the smoothing cost function The calculation formula is as follows:

[0031]

[0032] Where, β act N represents the smoothing scaling factor. act a represents the step length for summing actions. i Let t represent the output action of the navigator for i drones, and t represent the time step.

[0033] Optionally, the collision cost function The calculation formula is as follows:

[0034]

[0035] Where β obs N represents the collision scaling factor. obs Indicates the number of sensors, l obs L represents the sensor's maximum detection distance. n This represents the detected value of sensor n.

[0036] This invention also provides a drone target tracking system based on stochastic potential game theory, comprising:

[0037] A state parameter construction module is used to obtain the initial state of the UAV and the target in the simulation environment, and construct state parameters based on the initial state; the state parameters include: the relative position of the UAV and the target, the relative position of multiple UAVs, the observation value of the UAV and the observation value of the target;

[0038] A state training module is used to update the initial state based on the state parameters from the UAV navigator and the target navigator.

[0039] The cost function calculation module is used to calculate the cost function based on the updated state; the form of the cost function satisfies the conditions of a stochastic potential game.

[0040] An update gradient calculation module is used to calculate the update gradients of the UAV navigator and the target navigator based on the cost function and using a deep reinforcement learning framework;

[0041] A training module is used to train the UAV navigator and the target navigator based on the updated gradient;

[0042] The optimization module is used to deploy the updated drone navigator to the drone and optimize the trained drone navigator by obtaining the target state and drone state in the real scene.

[0043] The tracking module is used to track targets based on an optimized drone navigator.

[0044] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the above-described UAV target tracking method based on stochastic potential game.

[0045] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described UAV target tracking method based on stochastic potential game theory.

[0046] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0047] During the simulation phase, this invention trains the UAV navigator using simulation data, calculates the cost function based on a stochastic game strategy, and updates the navigator parameters using a deep reinforcement learning framework, enabling both parties' navigation decisions to reach a Nash equilibrium. In practical applications, this invention iteratively optimizes the UAV navigator periodically through observations of the UAV's state and the target's state in real-world scenarios, further enhancing its tracking capabilities. This invention enables multi-UAV systems to track and surround non-cooperative targets in real-time in unknown environments. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 The flowchart shows the UAV target tracking method based on stochastic potential game provided in Embodiment 1 of the present invention.

[0050] Figure 2 This is a flowchart of the training process for the UAV navigator and target navigator during the simulation phase, provided in Embodiment 1 of the present invention.

[0051] Figure 3 An optimized flowchart of the application stage of the UAV navigator provided in Embodiment 1 of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] The purpose of this invention is to provide a method, system, and device for UAV target tracking based on stochastic potential game theory. First, a training phase is completed in a simulation environment. Then, the trained UAV navigator is deployed to the actual UAV for application, enabling the UAV to track maneuvering targets. During application, the UAV navigator is dynamically updated and continuously improved based on the states of the UAV and the target. The algorithm is fully distributed, with each UAV possessing an independent navigator that makes decisions based on received state information. The term "intelligent agent" used below refers collectively to the various UAVs and the target in the game theory environment.

[0054] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Example 1

[0056] like Figure 1 As shown, the UAV target tracking method based on stochastic potential game theory provided in Embodiment 1 of the present invention includes the following steps:

[0057] S1: Obtain the initial state of the UAV and the target in the simulation environment, and construct state parameters based on the initial state; the state parameters include: the relative position of the UAV and the target, the relative position of multiple UAVs, the observation value of the UAV and the observation value of the target.

[0058] In practical applications, the multi-machine system and target state are initialized, and the starting position, heading, and speed of the UAV and target are selected by uniformly distributed random sampling method; the positions of no-fly zones or obstacles are initialized, and multiple no-fly zones of arbitrary shapes are randomly generated within the simulation area.

[0059] The simulated UAV and target state data, such as attitude and position data, are normalized and integrated to facilitate subsequent navigator updates. Therefore, [tl] n [t-1] consists of the relative positions of the UAV and the target over a continuous time period, the relative positional relationships between UAVs, and the environmental observations of each agent. n The duration of the saved state. Environmental observations refer to abstract sensor sampling data, such as detection data from airborne lidar, images captured by cameras, etc.

[0060] S2: Update the initial state based on the state parameters in the UAV navigator and the target navigator.

[0061] The navigator for both the UAV and the target can employ fully connected, recurrent, long short-term memory, or combined neural networks with multiple hidden layers, capable of fitting any nonlinear function. The input to the UAV navigator is state S. t The output is the desired heading and speed at the current moment. The airborne flight control algorithm calculates the control quantity based on the desired value. The flight control algorithm includes, but is not limited to, PID, sliding mode, and fuzzy controller. The UAV dynamics model calculates the UAV's state parameters for the next moment based on the control quantity, as shown in equation (1), where The parameters for the UAV's state at the next moment. The current state parameter, u uav f is the control variable at the current moment. A (·) represents the nonlinear dynamics model of the UAV.

[0062]

[0063] The target navigator input is state St It outputs navigation commands and updates the state parameters for the next time step based on the target dynamics model (similar to the UAV update method described above). The target can also possess a control algorithm. Based on the states of the multi-aircraft system and the target at time t+1, it updates the relative positional relationship and environmental observations to form...

[0064] S3: Calculate the cost function based on the updated state; the form of the cost function satisfies the conditions of a stochastic potential game.

[0065] The cost function for each agent is calculated based on equation (1), where β in each cost function term is a proportionality coefficient used to control its proportion in the total cost. The tracking cost function is defined as equation (2), where and V max These are the maximum ground speeds of the target and the drone, respectively. Let be the rate of change of the relative distance between UAV i and the target. Using the rate of change of distance as a variable in the cost function calculation is beneficial to guiding the navigator to converge toward the fastest tracking strategy.

[0066]

[0067]

[0068] The enclosing cost function is defined by equation (4), where The function defines the angle between the line connecting UAV i to the target and the line connecting UAV i+1 to the target. This function guides the multi-UAV system to gradually form an angle during the pursuit process. The formation with angular spacing enables multi-angle detection of the target.

[0069]

[0070] As shown in equation (5), The motion smoothing cost function is used to enhance the practicality of the algorithm. The expected speed and heading angular velocity output by the navigator need to be relatively smooth so that the flight control system at the back end can effectively and smoothly control the aircraft.

[0071]

[0072] Let L be the collision cost function, defined by equation (6), where L is the collision cost function. n For the sensor detection value, l obs For the maximum detection distance, N obs This represents the number of sensors.

[0073]

[0074] The cost function satisfies the conditions of stochastic potential game, that is, the cost function satisfies the derivative condition of equation (7). The partial derivative of the cost function of agent i with respect to its own action (the action is the navigator output) and the action of any agent j is equal to the partial derivative of the cost function of agent j with respect to the actions of i and j.

[0075]

[0076] S4: Based on the cost function, use a deep reinforcement learning framework to calculate the update gradients of the UAV navigator and the target navigator.

[0077] The navigator is updated using gradients calculated based on a deep reinforcement learning framework. This framework includes, but is not limited to, open-source algorithms such as MADDPG, MASAC, and FACMAC, which are designed for continuous action-state spaces and employ a "centralized learning, distributed execution" approach. Essentially, these algorithms are based on an improved Bellman optimal operator and backpropagation to optimize the agent's policy distribution. Bellman optimal operator B... * As shown in equation (8), V represents the median function V of the current deep reinforcement learning algorithm. i The update method, where γ is the discount rate, E[·] is the mathematical expectation, and R i Let be the cost function mentioned above.

[0078]

[0079] S5: Train the UAV navigator and the target navigator based on the updated gradient.

[0080] Based on the update gradient calculated in step S4, the navigators of each agent are updated through backpropagation. The value function update rate can be obtained by differentiating equation (8). The value function network and policy network are updated by combining the policy update rates of each algorithm with stochastic gradient descent methods such as SGD and Adam.

[0081] S6: Deploy the updated drone navigator onto the drone and optimize the trained drone navigator using the target state and drone state obtained from the real-world scenario.

[0082] S7: Target tracking is achieved based on an optimized drone navigator.

[0083] Steps S1-S5 above constitute the simulation phase, and steps S6-S7 constitute the application phase. The simulation phase is the training process for the UAV navigator and target navigator, such as... Figure 2 As shown, a UAV navigator and a target navigator are trained simultaneously in a simulation environment. The cost function value is calculated based on stochastic potential game theory and flight state, and the navigator weight parameters are updated using a deep reinforcement learning framework to bring the navigation decisions of both sides to a Nash equilibrium. The application stage involves the optimization process of the UAV navigator, such as... Figure 3 As shown, the trained UAV navigator is deployed on the aircraft and observes the target by applying it in a real-world scenario. The navigator parameters are updated online using the same Bellman operator as in the training phase.

[0084] Example 2

[0085] In order to implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a UAV target tracking system based on stochastic potential game is provided below.

[0086] The system includes:

[0087] A state parameter construction module is used to obtain the initial state of the UAV and the target in the simulation environment, and construct state parameters based on the initial state; the state parameters include: the relative position of the UAV and the target, the relative position of multiple UAVs, the observation value of the UAV and the observation value of the target;

[0088] A state training module is used to update the initial state based on the state parameters from the UAV navigator and the target navigator.

[0089] The cost function calculation module is used to calculate the cost function based on the updated state; the form of the cost function satisfies the conditions of a stochastic potential game.

[0090] An update gradient calculation module is used to calculate the update gradients of the UAV navigator and the target navigator based on the cost function and using a deep reinforcement learning framework;

[0091] A training module is used to train the UAV navigator and the target navigator based on the updated gradient;

[0092] The optimization module is used to deploy the updated drone navigator to the drone and optimize the trained drone navigator by obtaining the target state and drone state in the real scene.

[0093] The tracking module is used to track targets based on an optimized drone navigator.

[0094] Example 3

[0095] Embodiment 3 of the present invention provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the UAV target tracking method based on stochastic potential game provided in Embodiment 1.

[0096] In practical applications, the aforementioned electronic devices can be servers.

[0097] In practical applications, electronic devices include: at least one processor, memory, bus, and communication interface.

[0098] The processor, communication interface, and memory communicate with each other via a communication bus.

[0099] A communication interface is used to communicate with other devices.

[0100] The processor is used to execute programs, specifically the methods described in the above embodiments.

[0101] Specifically, the program may include program code, which includes computer operation instructions.

[0102] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The electronic device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0103] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk drive.

[0104] Example 4

[0105] Based on the description of Embodiment 3, Embodiment 4 of the present invention provides a storage medium on which a computer program is stored. The computer program can be executed by a processor to implement the UAV target tracking method based on stochastic potential game of Embodiment 1.

[0106] The UAV target tracking system based on stochastic potential game provided in Embodiment 2 of this invention exists in various forms, including but not limited to:

[0107] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0108] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access capabilities. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0109] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes: audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0110] (4) Other electronic devices with data interaction functions.

[0111] Specific embodiments of the subject matter have now been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing can be advantageous.

[0112] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0113] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this invention, the functions of each unit can be implemented in one or more software and / or hardware components. Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0117] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0118] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0119] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined in this invention, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0120] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0121] This invention can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules.

[0122] Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific transactions or implement specific abstract data types. This invention can also be practiced in distributed computing environments where transactions are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0123] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0124] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for tracking unmanned aerial vehicle (UAV) targets based on stochastic potential game theory, characterized in that, include: Obtain the initial state of the drone and the target in the simulation environment, and construct state parameters based on the initial state; The state parameters include: the relative position of the UAV and the target, the relative positions among multiple UAVs, the observation values ​​of the UAVs, and the observation values ​​of the target; The initial state is updated based on the state parameters, using both the UAV navigator and the target navigator. The cost function is calculated based on the updated state; the form of the cost function satisfies the conditions of a stochastic potential game. Based on the cost function, the update gradients of the UAV navigator and the target navigator are calculated using a deep reinforcement learning framework; The UAV navigator and the target navigator are trained based on the updated gradient; The trained drone navigator is deployed into the drone, and the trained drone navigator is optimized by obtaining the target state and drone state in the real scene. Target tracking is achieved based on an optimized drone navigator; The formula for calculating the cost function is as follows: in, Let represent the cost function of the i-th agent, which includes a drone and a target. Let i represent the tracking cost function of the i-th agent. Let i represent the encirclement cost function of the i-th agent. Let represent the action smoothing cost function of the i-th agent. Denotes the collision cost function of the i-th agent; The tracking cost function The calculation formula is as follows: in, Indicates the tracking ratio coefficient. Indicates the target's maximum ground speed. This indicates the maximum ground speed of the drone. Let N be the rate of change of the relative distance between UAV j and target k, and let N represent the total number of UAVs. The encirclement cost function The calculation formula is as follows: in, This represents the enclosing ratio factor. This represents the angle between the line connecting the i-th drone and the target and the line connecting the (i+1)-th drone and the target.

2. The UAV target tracking method based on stochastic potential game theory according to claim 1, characterized in that, Based on the state parameters, the initial state is updated using the UAV navigator and the target navigator, specifically including: The state parameters are input into the UAV navigator to obtain the expected values ​​of the UAV at the current moment; the expected values ​​include the expected heading and the expected speed. Based on the expected value, the control quantity is calculated using an airborne flight control algorithm; Based on the control variables, the UAV dynamics model is used to calculate the state of the UAV at the next moment; The status parameters are input into the target navigator to obtain navigation instructions; Based on the navigation instructions, the target's state at the next moment is calculated using a target dynamics model.

3. The UAV target tracking method based on stochastic potential game theory according to claim 1, characterized in that, The smoothing cost function The calculation formula is as follows: in, Indicates the smoothing ratio coefficient. Indicates the summation step length. Let t represent the output action of the navigator for the i-th drone, and t represent the time step.

4. The UAV target tracking method based on stochastic potential game theory according to claim 1, characterized in that, The collision cost function The calculation formula is as follows: in, This represents the collision ratio coefficient. Indicates the number of sensors. Indicates the sensor's maximum detection distance. This represents the detected value of sensor n.

5. A drone target tracking system based on stochastic potential game theory, characterized in that, The system is used to execute the UAV target tracking method based on stochastic potential game as described in any one of claims 1-4, and the system comprises: A state parameter construction module is used to obtain the initial state of the UAV and the target in the simulation environment, and construct state parameters based on the initial state; the state parameters include: the relative position of the UAV and the target, the relative position of multiple UAVs, the observation value of the UAV and the observation value of the target; A state training module is used to update the initial state based on the state parameters and the UAV navigator and the target navigator. The cost function calculation module is used to calculate the cost function based on the updated state; the form of the cost function satisfies the conditions of a stochastic potential game. An update gradient calculation module is used to calculate the update gradients of the UAV navigator and the target navigator based on the cost function and using a deep reinforcement learning framework; A training module is used to train the UAV navigator and the target navigator based on the updated gradient; The optimization module is used to deploy the trained drone navigator onto the drone and optimize the trained drone navigator by obtaining the target state and drone state in the real scene. The tracking module is used to track targets based on an optimized drone navigator.

6. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the UAV target tracking method based on stochastic potential game as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the UAV target tracking method based on stochastic potential game as described in any one of claims 1-4.

Images