UAV target selection method and device based on Markov game and Bayesian optimization
By building a drone target selection model based on Markov game and Bayesian optimization, the problem of dynamic adaptability and insufficient multi-objective processing capabilities of traditional countermeasures in complex confrontation scenarios is solved, and the intelligence and resource optimization of drone target selection are achieved.
Patent Information
- Application Number
- CN202510702335.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-28
- Publication Date
- 2025-08-22
- Estimated Expiration
- 2045-05-28
AI Technical Summary
Traditional countermeasures are not dynamically adaptable when facing complex confrontation scenarios, weak multi-objective processing capabilities, and high static reward functions and high computational complexity, making it difficult to effectively deal with the threat of black-flying drones.
Using a method based on Markov game and Bayesian optimization, the drone target feature data is extracted, the Markov game model is constructed and the Bayesian optimization reward function is defined. The Gaussian process is used for probabilistic modeling, and the drone target selection process is optimized.
It significantly improves the intelligence level and resource utilization efficiency of drones in target selection tasks, and provides reliable solutions for real-time decision-making in complex dynamic environments.
Smart Images
Figure CN120258326B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of drone countermeasure technology, and in particular to a drone target selection method and device based on Markov game and Bayesian optimization. Background Art
[0002] As the threat posed by illegal drones to public safety and other areas intensifies, traditional countermeasure technologies have problems such as insufficient dynamic adaptability, weak multi-target processing capabilities, static reward functions and high computational complexity. Existing solutions based on rules, traditional game theory, reinforcement learning and hardware upgrades are unable to effectively cope with complex confrontation scenarios due to their respective limitations. Summary of the Invention
[0003] Based on this, it is necessary to provide a UAV target selection method and device based on Markov game and Bayesian optimization to address the above technical problems.
[0004] A method for selecting a target for a UAV based on Markov game and Bayesian optimization, the method comprising:
[0005] Extracting target feature data of the UAV; the target feature data is obtained by splicing continuous historical data of the target UAV in the UAV field of view;
[0006] According to the distance between the UAV and the target UAV, the target feature data is subjected to sparse processing to generate a global representation;
[0007] Constructing a Markov game model; the action space of the Markov game model includes: counter-action and observation actions; the input of the action space is the global representation, which is used to calculate the probability distribution of executing each action under the current drone state; the policy network of the Markov game model outputs the action probability of each action based on the probability distribution; the reward function of the Markov game model includes the reward for the drone selecting the target and the state transition cost;
[0008] The reward function is defined as the objective function of Bayesian optimization, a Gaussian process is used as a proxy model to perform probabilistic modeling on the objective function, and an optimal output value of the reward function is obtained through iterative solution;
[0009] According to the optimal reward function output value, the drone target selection result of the Markov game model is output.
[0010] In one embodiment, the method further includes: continuously extracting characteristic data of the target drone for consecutive q steps of the drone, and continuously The characteristic data change value of the target UAV in the step;
[0011] According to step q, the characteristic data and the characteristic data change value are summed up in sequence to obtain a data splicing value;
[0012] The data splicing value is input into the Softmax function for normalization processing to obtain the target feature data of the drone.
[0013] In one embodiment, the method further includes: performing a sparse processing on the target feature data according to the distance between the UAV and the target UAV to generate a global representation as follows:
[0014] ;
[0015] in, Represents target feature data, , represents the target characteristics of target UAV i, A sparse attention mask vector representing the importance of all target drones.
[0016] In one embodiment, the method further includes: calculating the probability distribution of executing each action in the current state of the drone as follows:
[0017] ;
[0018] in, is the weight matrix, is the bias vector, Indicates the current status of the drone , actions in the action space The probability distribution when .
[0019] In one embodiment, the state transition cost is:
[0020] ;
[0021] ;
[0022] ;
[0023] ;
[0024] in, represents the state transition cost, Indicates execution of an action The cost of movement distance, Indicates the distance between the UAV and the target UAV, is the expected distance for the drone to counter the target drone, Indicates execution of an action The cost of uniform distribution is Indicates the number of friendly drones around the target selected by the drone at the current moment. is the number of friendly drones in the drone's field of view, Indicates the number of drones to be countered in the drone’s field of view. represents the target threat cost, Indicates the threat value of the target drone selected by the drone, Represents the weight parameter.
[0025] In one embodiment, the reward for the drone selecting a target is:
[0026] ;
[0027] in, is the threat impact value of the target, is the target distance impact value, The impact value of the number of friendly drones around the target, is the threat value importance parameter, is the target distance importance parameter, is the importance parameter of the number of friendly forces around the target, To control the state transfer cost degree of impact.
[0028] In one embodiment, the reward function is:
[0029] ;
[0030] in, represents the reward function, T represents the cumulative time, Express expectations.
[0031] In one embodiment, the method further includes: using a Gaussian process as a proxy model to perform probabilistic modeling on the objective function, including:
[0032] The objective function is rewritten to obey the Gaussian distribution as follows:
[0033] ;
[0034] ;
[0035] in, is the mean function, represents the squared exponential kernel function. represents the signal variance, is the length scale parameter.
[0036] In one embodiment, in each iteration, the posterior distribution of the Gaussian distribution is updated using the new data point as follows:
[0037] ;
[0038] in, is the predicted mean, D represents the training set input matrix, represents a normal distribution, is the prediction variance;
[0039] Construct the acquisition function as:
[0040] ;
[0041] ;
[0042] ;
[0043] in, Express expectations, is the currently known maximum objective function value, and are the cumulative distribution function and probability density function of the standard normal distribution, Expressing a desire for improvement;
[0044] According to the maximum acquisition function, select the next set of candidate weight combinations :
[0045] .
[0046] When the optimization process ends, the weight combination that maximizes the objective function value is output, otherwise the posterior distribution update is returned:
[0047] .
[0048] A UAV target selection device based on Markov game and Bayesian optimization, the device comprising:
[0049] A data extraction module is used to extract target feature data of the UAV; the target feature data is obtained by splicing the continuous historical data of the target UAV in the UAV field of view;
[0050] A global representation module, configured to perform a sparse processing on the target feature data according to the distance between the UAV and the target UAV to generate a global representation;
[0051] A model construction module is used to construct a Markov game model; the action space of the Markov game model includes: counter-action and observation actions. The input of the action space is the global representation, which is used to calculate the probability distribution of executing each action under the current drone state. The policy network of the Markov game model outputs the action probability of each action based on the probability distribution; the reward function of the Markov game model includes the reward for the drone to select the target and the state transition cost;
[0052] An optimization module, configured to define the reward function as an objective function of Bayesian optimization, perform probabilistic modeling of the objective function using a Gaussian process as a proxy model, and obtain an optimal output value of the reward function through iterative solution;
[0053] The output module is used to output the drone target selection result of the Markov game model according to the optimal reward function output value.
[0054] The aforementioned UAV target selection method and device, based on Markov games and Bayesian optimization, designs the action space and reward function for UAV target selection based on the Markov game framework. This method considers target characteristics and differential changes over time, extracts target feature information within the UAV's field of view, and inputs this information into a neural network to obtain the probability distribution of different UAV actions. A reward function, including state transition costs, is established to provide gradient information for policy network learning. Bayesian optimization is used to automatically adjust key weight parameters in the reward function to find the optimal strategy that maximizes cumulative rewards. This technology significantly improves the intelligence level and resource utilization efficiency of UAVs in target selection tasks, providing a reliable solution for real-time decision-making in complex dynamic environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 1 is a flow chart of a method for selecting a target for a UAV based on Markov game and Bayesian optimization in one embodiment;
[0056] Figure 2 The figure is a structural block diagram of a UAV target selection device based on Markov game and Bayesian optimization in one embodiment. DETAILED DESCRIPTION
[0057] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.
[0058] In one embodiment, Figure 1 As shown in FIG, a UAV target selection method based on Markov game and Bayesian optimization is provided, which includes the following steps:
[0059] Step 102: extract target feature data of the UAV.
[0060] The target feature data is obtained by splicing the continuous historical data of the target drone in the drone's field of view.
[0061] Step 104 : Sparsely process the target feature data according to the distance between the UAV and the target UAV to generate a global representation.
[0062] Step 106: construct a Markov game model.
[0063] The action space of the Markov game model includes: counter-action and observation actions. The input of the action space is the global representation, which is used to calculate the probability distribution of executing each action under the current drone state. The strategy network of the Markov game model outputs the action probability of each action based on the probability distribution; the reward function of the Markov game model includes the reward for the drone to select the target and the state transition cost.
[0064] In step 108, the reward function is defined as the objective function of Bayesian optimization, and the Gaussian process is used as a proxy model to perform probabilistic modeling on the objective function. The optimal output value of the reward function is obtained through iterative solution.
[0065] Step 110: Output the drone target selection result of the Markov game model according to the optimal reward function output value.
[0066] In the aforementioned drone target selection method based on Markov games and Bayesian optimization, the action space and reward function for drone target selection are designed based on the Markov game framework. The target characteristics and differential changes in target characteristics over a period of time are considered, and feature extraction is performed on the target feature information in the drone's field of view. This information is input into the neural network to obtain the probability distribution of different drone actions. A reward function that includes state transition costs is established to provide gradient information for policy network learning. Bayesian optimization is used to automatically adjust the key weight parameters in the reward function to find the optimal strategy that maximizes the cumulative reward. This technology significantly improves the intelligence level and resource utilization efficiency of drones in target selection tasks, providing a reliable solution for real-time decision-making in complex dynamic environments.
[0067] In one embodiment, the feature data of the target drone of the continuous q steps of the drone is continuously extracted, and the feature data of the target drone of the continuous q steps of the drone is continuously extracted. The characteristic data change value of the target UAV in the step q is summed up in sequence on the characteristic data and the characteristic data change value to obtain a data splicing value; the data splicing value is input into the Softmax function for normalization processing to obtain the target characteristic data of the UAV.
[0068] Specifically, the characteristic data includes threat value ,distance , Number of friendly drones , the feature data of all target drones are spliced into a historical data tensor , the characteristic data change values are respectively the threat value change value , distance change value 、Change in the number of friendly drones ,Finally, the target feature matrix :
[0069] ;
[0070] .
[0071] In one embodiment, the target feature data is sparsely processed based on the distance between the UAV and the target UAV to generate a global representation as follows:
[0072] ;
[0073] in, Represents target feature data, , represents the target characteristics of target UAV i, A sparse attention mask vector representing the importance of all target drones.
[0074] Specifically, because multiple drones may be in the drone's field of view, some of which are too far away to be suitable for countermeasures. When a drone detects only one target to be countered, it increases its attention weight to that target, regardless of its distance. When a drone detects multiple targets to be countered, it screens out drones that are too far away and ignores them using a sparse attention mechanism. This prevents distant drones from wasting the drone's onboard computing power, and increases the attention weight of drones within a certain range. Based on these rules, an attention mask for the drone target is generated.
[0075] In one embodiment, the action space includes maintaining observation and countermeasure targets ( Target correspondence The input of the action probability calculation is the global representation The probability distribution of executing each action in the current drone state is calculated as:
[0076] ;
[0077] in, is the weight matrix, is the bias vector, Indicates the current status of the drone , actions in the action space The probability distribution when .
[0078] The parameter update gradient is :
[0079] .
[0080] The policy network is realized through the above formula Learning based on reward functions, represents the reward function.
[0081] In one embodiment, the reward function includes the reward for drones selecting a target and the state transition cost. The state transition cost is typically composed of several core factors: movement distance cost (the distance the drone needs to move to counter the target in this state), distribution uniformity cost (the distribution effect of drones at the swarm level), and target threat cost (whether targets with higher threat levels are preferentially countered). These factors are combined into a total cost function through a weighted summation.
[0082] Specifically, the state transfer cost is:
[0083] ;
[0084] ;
[0085] ;
[0086] ;
[0087] in, represents the state transition cost, Indicates execution of an action The cost of movement distance, Indicates the distance between the UAV and the target UAV, is the expected distance for the drone to counter the target drone, Indicates execution of an action The cost of uniform distribution is Indicates the number of friendly drones around the target selected by the drone at the current moment. is the number of friendly drones in the drone's field of view, Indicates the number of drones to be countered in the drone’s field of view. represents the target threat cost, Indicates the threat value of the target drone selected by the drone, Represents the weight parameter. If the UAV does not change its state, then .
[0088] In another embodiment, based on the current state and actions , extract relevant features to calculate the reward function As the agent at each time step The immediate benefit is:
[0089] ;
[0090] in, is the threat impact value of the target, is the target distance impact value, The impact value of the number of friendly drones around the target, is the threat value importance parameter, is the target distance importance parameter, is the importance parameter of the number of friendly forces around the target, To control the state transfer cost degree of impact.
[0091] In one embodiment, the reward function is:
[0092] ;
[0093] in, represents the reward function, T represents the cumulative time, Express expectations.
[0094] The aforementioned drone target selection scheme involves numerous parameters. Bayesian optimization is used to automatically adjust key weights in the reward function, thereby finding the optimal strategy that maximizes the cumulative reward. This approach is particularly well-suited for optimization problems in high-dimensional parameter spaces. In drone target selection tasks, the design of the reward function often requires balancing multiple factors (threat value, distance, and the number of friendly forces around the target). Bayesian optimization can quickly find the optimal weight combination without the need for manual parameter tuning.
[0095] Specifically, the weight parameters directly affect the drone's target selection behavior. The following are the specific meanings of the weight parameters in the reward function and state transition cost:
[0096] :Control J distance cost the impact of;
[0097] :Control time cost the impact of;
[0098] :Control other factors impact.
[0099] Reward Function As the objective function, it is the UAV in a specific weight combination The following performance indicators;
[0100] .
[0101] Since evaluating the objective function requires running a simulation environment, which is typically computationally expensive, Bayesian optimization is used to reduce the number of evaluations. Gaussian processes are the most commonly used surrogate model in Bayesian optimization, used to probabilistically model the objective function.
[0102] In one embodiment, the objective function is rewritten to obey the Gaussian distribution as follows:
[0103] ;
[0104] ;
[0105] in, is the mean function, Represents the kernel function, which is used to describe the similarity between different weight combinations. Commonly used kernel functions include square exponential kernel, represents the signal variance, is the length scale parameter. The signal variance parameter and the length scale parameter are set empirically. In one embodiment, , .
[0106] In another embodiment, in each iteration, the posterior distribution of the Gaussian distribution is updated using the new data point as follows:
[0107] ;
[0108] in, is the predicted mean, D represents the training set input matrix, represents a normal distribution, is the prediction variance.
[0109] Specifically, in each iteration, the posterior distribution of the Gaussian distribution is updated using new data points:
[0110] : training set input matrix, ,in, is the i-th input vector, Represents the target value vector output by the i-th input vector;
[0111] : new input vector;
[0112] Update the posterior distribution and add new data points in each iteration After that, the posterior distribution is updated as:
[0113] .
[0114] To predict the mean, the update formula is:
[0115] ;
[0116] in, : The covariance matrix of the training points is defined as ;
[0117] : The covariance vector of the new point and the training set, defined as ;
[0118] : target value vector of the training set;
[0119] : Noise variance, which is an adjustable parameter. In one embodiment, .
[0120] : Identity matrix, maintain matrix positive definiteness;
[0121] : Prediction variance, calculated as:
[0122] .
[0123] Construct the acquisition function as:
[0124] ;
[0125] ;
[0126] ;
[0127] in, Express expectations, is the currently known maximum objective function value, and They are the cumulative distribution function and probability density function of the standard normal distribution, both of which are even functions. represents the expected improvement; the acquisition function is used to select the next set of candidate weight combinations to balance exploration and development.
[0128] Randomly select several groups of initial weight combinations in the weight space , and evaluate the objective function . Use the initial data points to build a Gaussian process surrogate model and calculate the predicted mean and the prediction variance , according to the acquisition function Maximum, select the next set of candidate weight combinations :
[0129] .
[0130] Run the simulation environment to evaluate the objective function value of the candidate weight combination . The new data point Add the dataset and repeat the above steps until the stopping condition (such as the maximum number of iterations or the convergence threshold) is met. When the optimization process ends, the weight combination that maximizes the objective function value is output. Otherwise, the posterior distribution update is returned:
[0131] .
[0132] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0133] In one embodiment, Figure 2 As shown, a UAV target selection device based on Markov game and Bayesian optimization is provided, comprising:
[0134] The data extraction module 202 is used to extract target feature data of the UAV; the target feature data is obtained by splicing the continuous historical data of the target UAV in the UAV field of view;
[0135] A global representation module 204 is configured to perform a sparse processing on the target feature data according to the distance between the UAV and the target UAV to generate a global representation;
[0136] Model construction module 206 is used to construct a Markov game model; the action space of the Markov game model includes: counter-action and observation actions. The input of the action space is the global representation, which is used to calculate the probability distribution of executing each action under the current drone state. The policy network of the Markov game model outputs the action probability of each action based on the probability distribution; the reward function of the Markov game model includes the reward for the drone to select the target and the state transition cost;
[0137] An optimization module 208 is configured to define the reward function as an objective function of Bayesian optimization, use a Gaussian process as a proxy model to perform probabilistic modeling on the objective function, and obtain an optimal output value of the reward function through iterative solution;
[0138] The output module 210 is used to output the drone target selection result of the Markov game model according to the optimal reward function output value.
[0139] The specific limitations of the UAV target selection device based on Markov game and Bayesian optimization can be found in the limitations of the UAV target selection method based on Markov game and Bayesian optimization described above and will not be further elaborated here. Each module in the aforementioned UAV target selection device based on Markov game and Bayesian optimization can be implemented in whole or in part via software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.
[0140] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0141] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0142] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and such modifications and improvements are intended to fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.
Claims
1. A UAV target selection method based on Markov game and Bayesian optimization, characterized in that: The method comprises: Extracting target feature data of the UAV; the target feature data is obtained by splicing continuous historical data of the target UAV in the UAV field of view; According to the distance between the UAV and the target UAV, the target feature data is subjected to sparse processing to generate a global representation; Constructing a Markov game model; the action space of the Markov game model includes: counter-action and observation actions; the input of the action space is the global representation, which is used to calculate the probability distribution of executing each action under the current drone state; the policy network of the Markov game model outputs the action probability of each action based on the probability distribution; the reward function of the Markov game model includes the reward for the drone selecting the target and the state transition cost; The reward function is defined as the objective function of Bayesian optimization, a Gaussian process is used as a proxy model to perform probabilistic modeling on the objective function, and an optimal output value of the reward function is obtained through iterative solution; Output the drone target selection result of the Markov game model based on the optimal reward function output value; The step of extracting target feature data of the UAV includes: Continuously extract the characteristic data of the target drone for the continuous q steps of the drone, and continuously The characteristic data change value of the target UAV in the step; According to step q, the characteristic data and the characteristic data change value are summed up in sequence to obtain a data splicing value; The data splicing value is input into the Softmax function for normalization processing to obtain the target feature data of the drone.
2. The UAV target selection method based on Markov game and Bayesian optimization according to claim 1 is characterized in that: Based on the distance between the UAV and the target UAV, the target feature data is sparsely processed to generate a global representation, including: According to the distance between the UAV and the target UAV, the target feature data is sparsely processed to generate a global representation: in, Represents target feature data, , represents the target characteristics of target UAV i, A sparse attention mask vector representing the importance of all target drones.
3. The UAV target selection method based on Markov game and Bayesian optimization according to claim 2 is characterized in that: Calculate the probability distribution of executing each action under the current drone state, including: The probability distribution of executing each action under the current drone state is calculated as: in, is the weight matrix, is the bias vector, Indicates the current status of the drone , actions in the action space The probability distribution when .
4. The UAV target selection method based on Markov game and Bayesian optimization according to claim 3 is characterized in that: The state transfer cost is: in, represents the state transition cost, Indicates execution of an action The cost of movement distance, Indicates the distance between the UAV and the target UAV, is the expected distance for the drone to counter the target drone, Indicates execution of an action The cost of uniform distribution is Indicates the number of friendly drones around the target selected by the drone at the current moment. is the number of friendly drones in the drone's field of view, Indicates the number of drones to be countered in the drone’s field of view. represents the target threat cost, Indicates the threat value of the target drone selected by the drone, Represents the weight parameter.
5. The UAV target selection method based on Markov game and Bayesian optimization according to claim 4 is characterized in that: The reward for the drone to select a target is: in, is the threat impact value of the target, is the target distance impact value, The impact value of the number of friendly drones around the target, is the threat value importance parameter, is the target distance importance parameter, is the importance parameter of the number of friendly forces around the target, To control the state transfer cost degree of impact.
6. The UAV target selection method based on Markov game and Bayesian optimization according to claim 5 is characterized in that: The reward function is: in, represents the reward function, T represents the cumulative time, Express expectations.
7. The UAV target selection method based on Markov game and Bayesian optimization according to claim 6 is characterized in that: The objective function is probabilistically modeled using a Gaussian process as a proxy model, including: The objective function is rewritten to obey the Gaussian distribution as follows: in, is the mean function, represents the kernel function, represents the weight parameter combination, represents the signal variance, is the length scale parameter.
8. The UAV target selection method based on Markov game and Bayesian optimization according to claim 7 is characterized in that: The optimal reward function output value is obtained through iterative solution, including: In each iteration, the posterior distribution of the Gaussian distribution is updated with the new data point: in, is the predicted mean, D represents the training set input matrix, represents a normal distribution, is the prediction variance; Construct the acquisition function as: in, Express expectations, is the currently known maximum objective function value, and are the cumulative distribution function and probability density function of the standard normal distribution, Expressing a desire for improvement; According to the maximum acquisition function, select the next set of candidate weight combinations : When the optimization process ends, the weight combination that maximizes the objective function value is output, otherwise the posterior distribution update is returned: 。 9. A UAV target selection device based on Markov game and Bayesian optimization, used to implement the UAV target selection method based on Markov game and Bayesian optimization according to any one of claims 1 to 8, characterized in that: The device comprises: A data extraction module is used to extract target feature data of the UAV; the target feature data is obtained by splicing the continuous historical data of the target UAV in the UAV field of view; A global representation module, configured to perform a sparse processing on the target feature data according to the distance between the UAV and the target UAV to generate a global representation; A model construction module is used to construct a Markov game model; the action space of the Markov game model includes: counter-action and observation actions. The input of the action space is the global representation, which is used to calculate the probability distribution of executing each action under the current drone state. The policy network of the Markov game model outputs the action probability of each action based on the probability distribution; the reward function of the Markov game model includes the reward for the drone to select the target and the state transition cost; An optimization module, configured to define the reward function as an objective function of Bayesian optimization, perform probabilistic modeling of the objective function using a Gaussian process as a proxy model, and obtain an optimal output value of the reward function through iterative solution; The output module is used to output the drone target selection result of the Markov game model according to the optimal reward function output value.