Complex scene strategy generation method and system based on decision tree and UCB algorithm
The policy generation method combining decision trees and the UCB algorithm solves the problem of policy generation in complex scenarios for intelligent decision-making systems, improving policy diversity and adaptability, and enhancing the accuracy and precision of policy evaluation.
Patent Information
- Application Number
- CN202511498934.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-20
- Publication Date
- 2026-02-06
AI Technical Summary
Existing intelligent decision-making systems struggle to balance strategy exploration and utilization in complex and dynamic scenarios, resulting in insufficient diversity in strategy generation and a lack of refinement in the evaluation system, which affects the accuracy and adaptability of strategy generation.
A strategy generation method based on decision trees and UCB algorithm is adopted. An initial strategy set is generated by decision trees, the optimal strategy is screened by UCB algorithm, and the auction price is calculated based on the adjustment plan for comprehensive evaluation. Finally, the optimal strategy is selected.
It achieves enhanced diversity and adaptability in strategy selection under complex scenarios, improves the precision and accuracy of strategy evaluation, and can dynamically balance the utilization and exploration of high-scoring and low-frequency strategies, avoiding the decision-making process from being fixed on a limited number of options.
Smart Images
Figure CN121481684A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a method and system for generating strategies for complex scenarios based on decision trees and UCB algorithms. Background Technology
[0002] In intelligent decision-making systems for fields such as airspace security, financial risk control, and industrial operations, a core challenge is the ability to automatically generate and evaluate optimal action strategies under complex conditions of limited resources and rapidly changing environments. Strategy generation is a crucial component of intelligent decision-making systems, automatically deriving and constructing feasible task plans or action sequences under given objectives and external constraints. For example, in airspace security missions, facing intrusions from multiple drones at varying speeds and altitudes, the system needs to combine detection, interception, and jamming resources to quickly generate multiple executable response plans. Simultaneously, strategy evaluation is equally indispensable. It involves comprehensively analyzing candidate solutions from dimensions such as effectiveness, cost, and robustness, assessing the success rate of execution, resource consumption, and stability under uncertain environments, thereby selecting the optimal or near-optimal execution strategy from multiple candidate options.
[0003] In scenarios such as airspace security, financial risk control, and industrial operation and maintenance, facing the challenges of an ever-increasing number of rules and rapidly changing environmental conditions, research and engineering practices on intelligent decision-making and dynamic strategy optimization have mainly formed several representative methods: The earliest approach relied on static rule bases, manually defining the mapping between conditions and actions, and directly matching and invoking them when a task was triggered. This method was simple to implement, quick to deploy, and effective when the scenario was limited in scale and the environment was relatively stable. However, as task complexity increased, this method became increasingly unable to handle the uncertainties brought about by cross-departmental, multi-source data and dynamic environments.
[0004] Subsequently, methods based on the fusion of statistical and probabilistic models were developed. These methods assign confidence levels or weights to rules from different sources, and then select the optimal rule through weighted averaging or voting mechanisms. These methods take into account the differences in data sources to some extent and are more flexible than a simple static rule base, but their effectiveness is highly dependent on the distribution of historical data. Once the data is sparse or the environment changes abruptly, evaluation bias is likely to occur, making it difficult to guarantee stability.
[0005] In recent years, decision trees and closed-loop feedback mechanisms have been gradually introduced into intelligent decision-making systems to address issues such as the separation of policy generation and feedback, low efficiency of policy selection, insufficient accuracy of evaluation, and low intelligence of feedback in traditional systems. However, intelligent decision-making systems based on decision trees and closed-loop feedback mechanisms still have many shortcomings in complex dynamic scenarios. The policy selection mechanism relies too much on the accumulation of historical scores, only selects the strategy with the highest historical score, and then supplements the strategy to form a candidate strategy set through a simple random method. This approach lacks a mechanism to explore strategies with low historical scores but high potential value, and it is prone to path dependence, leading to insufficient strategy diversity. Additionally, the evaluation system only focuses on general dimensions of radar tasks, ignores the actual cost differences between different resources, and does not combine with scene parameters for refinement, which severely affects the accuracy of strategy evaluation and the accuracy of strategy generation.
[0006] Therefore, there is an urgent need for a strategy generation method that can balance strategy exploration and utilization and has the ability to fine-tune scene evaluation to improve the diversity, adaptability, and precision of intelligent decision-making. SUMMARY
[0007] To solve the above problems existing in the prior art, the present application provides a complex scene strategy generation method and system based on decision trees and UCB algorithms.
[0008] The technical problems to be solved by the present application are solved by the following technical solutions: In a first aspect, the present application provides a complex scene strategy generation method based on decision trees and UCB algorithms, comprising: Step 10: Obtain task scene data; Step 20: Generate an initial strategy set based on the task scene data through a pre-constructed decision tree; Step 30: Use the UCB algorithm to select the best strategy from the initial strategy set to obtain a feasible strategy set; ≥1; Step 40: Generate an adjustment scheme for each strategy in the feasible strategy set based on the task scene data, determine the auction price of the strategy based on the adjustment scheme, and select the optimal strategy based on the auction price; the adjustment scheme is used to map resources to targets for the strategy; the auction price is used to comprehensively evaluate the ability of the adjustment scheme in terms of time cost, execution cost, and execution effect; Step 50: Execute the adjustment scheme corresponding to the optimal strategy, calculate the strategy execution score based on the execution result, and update the historical performance data of the optimal strategy based on the strategy execution score.
[0009] Optionally, the step 30 specifically comprises: computing a value of each strategy in the initial strategy set wherein represents a value of the i-th strategy, represents a historical average task effect score of the i-th strategy, represents an exploration potential score, represents a preset exploration factor, represents a historical execution number of the i-th strategy, represents a total historical execution number of all strategies in the initial strategy set. selecting, from the initial strategy set, a number of strategies with the highest values, to obtain a feasible strategy set; the number of strategies is determined according to the following formula: wherein represents a number of strategies in the initial strategy set.
[0010] Optionally, the step 40 specifically comprises: Step 41: generating an adjustment scheme of each strategy in the feasible strategy set according to a strategy type of each strategy in the feasible strategy set, by a rule mapping or an optimization algorithm, and monitoring a generation time consumption of the adjustment scheme of each strategy, and terminating the generation of the adjustment scheme of the strategy when the generation time consumption exceeds a preset timeout threshold. Step 42: calculating an auction price of the strategy according to the adjustment scheme: wherein represents an index of the strategy corresponding to the adjustment scheme, represents the auction price of the i-th strategy, represents a time cost score of the i-th strategy, represents an execution cost score of the i-th strategy, represents an execution effect score of the i-th strategy, Step 43: taking the strategy with the highest auction price as an optimal strategy.
[0011] Optionally, the time cost score is calculated as follows: ; in, Indicates the first The time required to generate adjustment plans for each strategy. This represents the maximum time required to generate adjustment schemes for all strategies; The execution cost score is calculated as follows: ; in, , Indicates the first The implementation cost of adjusting the strategy. Indicates the first Each strategy adjustment plan utilizes a set of resources. Representing resources unit cost Representing resources The number of times it is used This represents the maximum execution cost of all strategy adjustment plans; The execution performance score is calculated as follows: ; in, Indicates the index of the evaluation dimension. Representing resources In the Scores for each evaluation dimension Indicates the first The weights of each evaluation dimension, Indicates the number of evaluation dimensions. Indicates the first The total number of resources used by each strategy adjustment plan.
[0012] Optionally, the method further includes: normalizing the auction price so that the strategy with the highest normalized auction price is the optimal strategy; The specific method for normalizing the auction price is as follows: ; in, This represents the normalized auction price.
[0013] Optionally, the step of calculating the strategy execution score based on the execution result and updating the historical performance data of the optimal strategy based on the strategy execution score includes: Obtain the task success rate and strategy execution time based on the execution results; Calculate the optimal strategy execution score based on the task success rate and strategy execution time: ; in, Indicates the strategy execution score. Indicates the task success rate. The weights representing the success rate of the task. , Indicates the time factor. Indicates the strategy execution time. This indicates the preset baseline strategy time. Indicates multiplication operation; Update the historical execution count and historical average task performance score of the optimal strategy based on the strategy execution score.
[0014] Optionally, the update method for the historical execution count and the historical average task performance score is as follows: ; ; in, This represents the historical average task performance score of the optimal strategy. This represents the historical execution count of the optimal strategy. This represents the updated historical average task performance score. This indicates the number of times the code has been executed in the updated history.
[0015] Secondly, this invention provides a complex scene policy generation system based on decision tree and UCB algorithm, comprising: The acquisition module is used to acquire task scenario data; The strategy generation module is used to generate an initial strategy set based on the task scenario data using a pre-built decision tree. The strategy selection module is used to select the optimal strategy from the initial strategy set using the UCB algorithm. One strategy is used to obtain a set of feasible strategies; ≥1; The adjustment scheme generation and evaluation module is used to generate an adjustment scheme for each strategy in the feasible strategy set based on the task scenario data, determine the auction price of the strategy based on the adjustment scheme, and select the optimal strategy based on the auction price; the adjustment scheme is used to map resources and objectives for the strategy; the auction price is used to comprehensively evaluate the capabilities of the adjustment scheme in three dimensions: time cost, execution cost, and execution effect; The strategy evaluation and feedback module is used to execute the adjustment plan corresponding to the optimal strategy, calculate the strategy execution score based on the execution result, and update the historical performance data of the optimal strategy based on the strategy execution score.
[0016] The application provides a complex scene strategy generation method based on a decision tree and a UCB algorithm.
[0017] The application will be further described in detail below in combination with the drawings and the application. BRIEF DESCRIPTION OF DRAWINGS
[0018] Figure 1 is a flowchart of a complex scene strategy generation method based on a decision tree and a UCB algorithm provided by an embodiment of the application; Figure 2 is a flowchart of another complex scene strategy generation method based on a decision tree and a UCB algorithm provided by an embodiment of the application; Figure 3 is a structural diagram of a complex scene strategy generation system based on a decision tree and a UCB algorithm provided by an embodiment of the application. DETAILED DESCRIPTION
[0019] The application will be further described in detail below in combination with specific embodiments, but the embodiments of the application are not limited thereto.
[0020] In order to achieve the balance between strategy stability and exploration of the intelligent strategy generation method, the actual cost difference of different scenes is fully considered, fine scene strategy evaluation is realized, and the diversity, adaptability and precision of intelligent decision are improved, an embodiment of the application provides a complex scene strategy generation method and system based on a decision tree and a UCB algorithm.
[0021] First, the complex scene strategy generation method based on a decision tree and a UCB algorithm provided by an embodiment of the application is described in detail. Referring to Figure 1 and Figure 2 , the method comprises the following steps: Step 10: obtaining task scene data.
[0022] Specifically, the task scene data is obtained to represent the current task scene, and the input is provided for strategy generation. Exemplarily, the task scene data includes a task target, a list of available resources, and environmental parameters. For example, in the UAV denial scene, the task scene data includes defense zone information, target information, and resource information.
[0023] Step 20: According to the task scene data, an initial strategy set Pi_M is generated by a pre-constructed decision tree.
[0024] Here, different decision trees are pre-constructed for different task scenes, and the adapted decision tree is called for strategy generation through the root node selection in application. The specific process is as follows: according to the task scene data, the pre-constructed decision tree is traversed from the root node, the rules in the condition nodes are matched through the task scene data, the branch path is selected layer by layer, and one or more leaf nodes are reached, each leaf node is associated with a pre-defined strategy (such as rule 1 “distance priority” or algorithm 1 “genetic algorithm”); finally, the strategies associated with these leaf nodes are collected to generate an initial strategy set Pi_M containing M feasible strategies.
[0025] In an implementation mode, the method further includes: periodically analyzing historical decision data, mining better decision rules, and updating and reconstructing the decision tree to realize continuous iteration and optimization of decision quality.
[0026] Step 30: The UCB algorithm is used to select the optimal strategy from the initial strategy set Pi_M, and a feasible strategy set Pi_N is obtained. ≥1.
[0027] Specifically, the core idea of the UCB (Upper Confidence Bound) algorithm is to balance “exploration” and “exploitation” through a mathematical formula. The algorithm does not simply select the option with the highest historical average return, but calculates a “confidence upper bound” score for each option. The score is composed of two parts: “average return” and “exploration reward”. The former encourages the use of known high-quality options, and the latter tends to try options that are insufficiently sampled but have great potential.
[0028] Here, the step specifically includes: (1) The value of each strategy in the initial strategy set Pi_M is calculated. Wherein, the value of the i-th strategy is denoted as The historical average task performance score of each strategy Indicates the potential score for exploration. This represents the preset exploration factor. Indicates the first The historical number of times each strategy has been executed. This represents the total number of historical executions of all strategies in the initial strategy set.
[0029] here, For use item, As an exploratory term, an exploratory term ensures that strategies with fewer executions still have a chance to be selected. This is used to balance exploration and utilization.
[0030] (2) Select from the initial policy set Pi_M The highest value We have several strategies, resulting in a feasible strategy set Pi_N; where... The calculation method obtained through adaptive computation is as follows: ; in, This indicates the number of policies in the initial policy set.
[0031] In one implementation, when all policies in the initial policy set Pi_M... If all values are equal or all are 0, then select randomly. The strategies constitute a feasible strategy set Pi_N.
[0032] This invention uses the UCB algorithm to select a feasible strategy set Pi_N, which allows for the utilization of historically efficient strategies while providing appropriate exploration opportunities for low-frequency strategies. This ensures the stability of existing high-scoring strategies while continuously uncovering potential high-value solutions, preventing the decision-making process from becoming stuck on a limited set of options. This balancing mechanism effectively enhances the diversity and adaptability of strategy selection.
[0033] Step 40: Based on the task scenario data, generate adjustment plans for each strategy in the feasible strategy set Pi_N, determine the auction price of the strategy based on the adjustment plan, and select the optimal strategy based on the auction price. Here, the adjustment plan is used to map resources and objectives to the strategy; the auction price is used to comprehensively evaluate the capabilities of the adjustment plan in terms of time cost, execution cost, and execution effect.
[0034] Specifically, for each strategy in the feasible strategy set Pi_N, an adjustment plan (i.e., resource-objective mapping) is generated based on the strategy type and task scenario data. The auction price of the strategy is then calculated based on the adjustment plan. Finally, the strategy with the highest auction price among all strategies is selected as the optimal strategy, and its adjustment plan is output. This step specifically includes: Step 41: Based on the policy type of each policy in the feasible policy set Pi_N, generate an adjustment plan for each policy through rule mapping or optimization algorithm, and monitor the generation time of the adjustment plan for each policy. When the generation time exceeds the preset timeout threshold, terminate the generation of the adjustment plan for that policy.
[0035] Specifically, when generating adjustment schemes for strategies, the matching relationship between resources and targets is calculated based on the strategy type (rule-based or algorithm-based) to achieve resource-target mapping. For rule-based strategies, the system performs mapping using predefined deterministic rules; for algorithm-based strategies, the corresponding optimization algorithm is invoked to solve the problem. For example, the "distance-first" strategy selects the resource closest to the target by calculating the distance between the resource and the target; the "genetic algorithm" strategy aims to minimize the total distance and maximize the success rate by iteratively searching to obtain the optimal mapping relationship.
[0036] Step 42: Calculate the auction price based on the adjustment plan: ; in, Indicates the index of the strategy corresponding to the adjustment plan. Indicates the first The auction price for each strategy, Indicates the first The time cost score for each strategy Indicates the first The execution cost score of each strategy. Indicates the first The score for the effectiveness of each strategy. , and These represent the weights for time cost, execution cost, and execution effect, respectively.
[0037] Here, the time cost score is calculated as follows: ; in, Indicates the first The time required to generate adjustment plans for each strategy. This represents the maximum time required to generate adjustment schemes for all strategies; The execution cost score is calculated as follows: ; in, , Indicates the first The implementation cost of adjusting the strategy. Indicates the first Each strategy adjustment plan utilizes a set of resources. Representing resources unit cost Representing resources The number of times it is used This represents the maximum execution cost of all strategy adjustment plans; The performance score is calculated as follows: ; in, Indicates the index of the evaluation dimension. Representing resources In the Scores for each evaluation dimension Indicates the first The weights of each evaluation dimension, Indicates the number of evaluation dimensions. Indicates the first The total number of resources used by each strategy adjustment plan. Here, Different calculation functions are used depending on the resource type and target attributes.
[0038] Step 43: Select the strategy with the highest auction price as the optimal strategy.
[0039] Specifically, select the strategy with the highest auction price from all strategies as the optimal strategy, output the optimal strategy and its adjustment plan, and complete the task by executing the adjustment plan.
[0040] Optionally, in one implementation, before selecting the optimal strategy, the method further includes: normalizing the auction price so that the strategy with the highest normalized auction price is selected as the optimal strategy. For example, the auction price is mapped to the (0, 100) interval using the Sigmoid function, specifically as follows: ; in, This represents the normalized auction price.
[0041] Understandably, this invention constructs a multi-dimensional evaluation system for complex scenarios, uniformly quantifying indicators such as time cost, execution cost, execution effect, and environmental adaptability. Furthermore, this invention refines the execution cost and execution effect, making it more adaptable to the differences in objectives across various task scenarios. This ensures the scientific validity, rationality, and interpretability of the evaluation results, improving the accuracy of strategy evaluation and the precision of strategy decision-making. For example, for four types of countermeasure resources in UAV countermeasure scenarios, such as communication jammers and navigation decoys, five dimensions—distance, altitude, speed, size, and visibility—are designed to accurately quantify the effectiveness of different resources in specific scenarios.
[0042] Step 50: Execute the adjustment plan corresponding to the optimal strategy, calculate the strategy execution score based on the execution result, and update the historical performance data of the optimal strategy based on the strategy execution score.
[0043] Specifically, after each task is completed, the execution result data of the strategy execution is collected and the strategy execution score is calculated. Then, based on the strategy execution score, the key parameters in the UCB algorithm are updated: the historical average task effect score and the historical execution count, which in turn affect the subsequent strategy generation and optimization process.
[0044] Here, the strategy execution score is calculated based on the execution results, and the historical performance data of the optimal strategy is updated based on the strategy execution score, including: First, obtain the task success rate and strategy execution time based on the execution results.
[0045] Secondly, based on the task success rate and strategy execution time, calculate the strategy execution score of the optimal strategy: ; in, Indicates the strategy execution score. Indicates the task success rate. The weights representing the success rate of the task. , Indicates the time factor. Indicates the strategy execution time. This indicates the preset baseline strategy time. Indicates multiplication operation; Understandable, generally speaking The weight should be greater than that of the time factor. This highlights the dominant role of task completion in the overall evaluation; the time factor reflects time efficiency through an exponential decay function, where the shorter the strategy execution time, the higher the strategy execution score, and when the completion time exceeds the baseline time, the strategy execution score decays exponentially.
[0046] Finally, the historical execution times and the historical average task effect score of the optimal strategy are updated according to the strategy execution score.
[0047] Exemplarily, in an implementation manner, the updating manners of the historical execution times and the historical average task effect score are as follows: ; ; wherein, the historical average task effect score of the optimal strategy, the historical execution times of the optimal strategy, the updated historical average task effect score, the updated historical execution times.
[0048] It can be understood that through the closed-loop mechanism of “execution-evaluation-update-optimization”, the method can continuously correct and evolve itself in the running process, and long-term maintain high matching with the complex dynamic environment, so that the overall improvement is realized in the aspects of decision efficiency, result scientificity and system adaptability. Moreover, the application discards the fixed reward logic, and directly uses the strategy execution score to update the “exploitation term” in the UCB algorithm, so that the subtle score difference can affect the subsequent strategy selection, and the marginal decision value of the score is embodied.
[0049] The complex scene strategy generation method based on the decision tree and the UCB algorithm provided by the application generates an initial strategy set through the decision tree, then filters the strategies in the initial strategy set to obtain a feasible strategy set by using the UCB algorithm according to the historical performance data and the exploration potential of the strategies, generates an adjustment scheme of each strategy in the feasible strategy set, calculates a strategy auction price according to the adjustment scheme, and selects an optimal strategy according to the auction price. By introducing the UCB algorithm in the strategy selection stage, the strategy selection is performed based on the historical performance data and the exploration potential of the strategies. This mechanism makes the dynamic balance between “exploiting” high-score strategies and “exploring” low-frequency strategies, which can guarantee the stability of the existing high-score strategies and continuously excavate potential high-value schemes, avoid the long-term fixation of the decision process on limited options, and improve the adaptability of the complex scene.
[0050] In addition, the application constructs a multi-dimensional evaluation system for complex scenes, evaluates the strategies from the dimensions of time cost, execution cost and execution effect, and refines the execution cost and the execution effect, so that the evaluation system is more suitable for the target differences of different task scenes, thereby guaranteeing the scientificity, rationality and explainability of the evaluation results, improving the evaluation precision of the strategies, and improving the accuracy of the strategy decision.
[0051] Next, the unmanned aerial vehicle denial scenario is taken as an example to further illustrate the complex scene strategy generation method based on the decision tree and the UCB algorithm of the embodiment of the application.
[0052] Scenario background: In a certain unmanned aerial vehicle defense system, two unmanned aerial vehicle intrusion events are detected, and the intelligent decision system needs to quickly generate effective countermeasures.
[0053] The task scene data includes: defense zone information, target unmanned aerial vehicle information and available resource information; specifically as follows: Defense zone information: 2 defense zones, radius 2000 meters, high visibility; Target information: 2 unmanned aerial vehicles, including: Target 901: medium size (3), speed 14.5 m / s, height 150 meters; Target 903: small size (2), speed 37.3 m / s, height 102.4 meters; Resource information: 8 countermeasures, specifically including: Communication jammer: 2 (201, 203); Laser weapon: 2 (202, 204); Surrounding unmanned aerial vehicle: 2 (205, 206); Navigation decoy: 2 (207 unavailable, 208 available).
[0054] The specific execution steps of the method of the application in this task scene are as follows: First step: obtaining task scene data.
[0055] Second step: generating an initial strategy set Pi_M according to the task scene data through the pre-constructed decision tree.
[0056] Here, the decision tree generates an initial strategy set Pi_M according to the scene characteristics, which contains 6 strategies in total: Rule 1 (distance priority): strategy ID 10001; Rule 2 (deception priority + distance priority): strategy ID 10002; Rule 3 (interference priority + distance priority): strategy ID 10003; Rule 4 (capture priority + distance priority): strategy ID 10004; Rule 5 (destroy priority + distance priority): strategy ID 10005; Algorithm 1 (genetic algorithm): strategy ID 10006.
[0057] Third step: screening the optimal strategy from the initial strategy set Pi_M by using the UCB algorithm. A strategy, get a feasible strategy set Pi_N; =2.
[0058] Here, for each strategy in the initial strategy set Pi_M, according to the historical performance data of the strategy (historical average task effect score and historical execution times), the strategy value is calculated, and the 2 strategies with the highest value are selected to form the feasible strategy set Pi_N. In this embodiment, strategies 10001 and 10003 are selected.
[0059] Fourth step: According to the task scene data, generate an adjustment scheme for each strategy in the feasible strategy set Pi_N, and determine the auction price of the strategy according to the adjustment scheme, and filter the optimal strategy according to the auction price. Specifically as follows: Strategy 10001 (distance priority) adjustment scheme: Target 901 -> resource 206 (surrounding unmanned aerial vehicle), distance 994.75 meters; Target 903 -> resource 202 (laser weapon), distance 271.63 meters; Adjustment scheme generation time: 199ms; countermeasure execution cost: 900.0; countermeasure execution effect score: 4.99; auction price: 0.6658.
[0060] Strategy 10003 (interference priority + distance priority) adjustment scheme: Target 901 -> resource 206 (surrounding unmanned aerial vehicle), distance 994.75 meters; Target 903 -> resource 203 (communication jammer), distance 299.88; Adjustment scheme generation time: 32ms; countermeasure execution cost: 700.0; countermeasure execution effect score: 5.31; auction price: 0.7126.
[0061] According to the above adjustment scheme and various index values, the auction price of each strategy is calculated as follows: Strategy 10003: Auction price 0.7126; Strategy 10001: Auction price 0.6658.
[0062] Therefore, the strategy 10003 with the highest auction price is taken as the optimal strategy, and the adjustment scheme of the strategy 10003 is taken as the final execution scheme.
[0063] Here, taking strategy 10003 as an example, the specific calculation process of the auction price is as follows: ; ; ; wherein, score the execution effect of the resource 206 on the target 901, score the execution effect of the resource 203 on the target 903; and each from the five evaluation dimensions of distance, height, speed, size and visibility. For example: The score of each dimension is: the distance score is 10 x (1-min (1, 299.88 / 500)) = 4.002, the height score is 8 (basically not affected), the speed score is 10 (not sensitive to speed), the size score is 7 (no differential interference), and the visibility score is 10 (all-weather work).
[0064] ; wherein the normalization factor is 4.918; .
[0065] Fifth step: execute the adjustment scheme corresponding to the strategy 10003, calculate the strategy execution score according to the execution result, and update the historical performance data of the strategy 10003 according to the strategy execution score.
[0066] Suppose the execution result is: the number of successfully counteracting unmanned aerial vehicles: 2, the total number of targets: 2, the strategy execution time: 180 seconds, and the benchmark strategy time : 300 seconds.
[0067] According to the above execution result data, the strategy execution score is calculated: task success rate: , that is, the task success rate is 100%; time factor: ; strategy execution score: ; After that, according to the strategy execution score, the UCB parameters of the strategy 10003 are updated: the historical average task effect score and the historical execution times: ; .
[0068] This embodiment demonstrates that the UCB algorithm strikes a balance between "utilization" (selecting historically successful strategies) and "exploration" (giving unused strategies a chance). Furthermore, it comprehensively evaluates the capabilities of adjustment schemes across three dimensions: time cost, execution cost, and execution effectiveness, using auction prices. For four types of countermeasure resources in UAV countermeasure scenarios, including communication jammers and navigation decoys, it designs five evaluation dimensions—distance, altitude, speed, size, and visibility—for scoring (e.g., scoring for net-capturing UAVs is limited to within 2km, and high-energy laser scoring varies significantly with visibility)—precisely quantifying the effectiveness of different resources in specific scenarios, thereby selecting the optimal strategy. This embodiment proves the effectiveness and practicality of the method in complex UAV denial scenarios. The method can quickly generate high-quality strategy solutions and continuously improve decision-making quality through a learning mechanism.
[0069] Corresponding to the above-described method for generating complex scene policies based on decision trees and UCB algorithms, this invention also provides a system for generating complex scene policies based on decision trees and UCB algorithms; such as Figure 3 As shown, the system may include: Module 301 is used to acquire task scenario data; The strategy generation module 302 is used to generate an initial strategy set based on the task scenario data through a pre-constructed decision tree; The strategy selection module 303 is used to select the optimal strategy from the initial strategy set using the UCB algorithm. One strategy is used to obtain a set of feasible strategies; ≥1; The adjustment scheme generation and evaluation module 304 is used to generate an adjustment scheme for each strategy in the feasible strategy set based on the task scenario data, determine the auction price of the strategy based on the adjustment scheme, and select the optimal strategy based on the auction price; the adjustment scheme is used to map resources and objectives for the strategy; the auction price is used to comprehensively evaluate the capabilities of the adjustment scheme in three dimensions: time cost, execution cost, and execution effect.
[0070] The strategy evaluation and feedback module 305 is used to execute the adjustment plan corresponding to the optimal strategy, calculate the strategy execution score based on the execution result, and update the historical performance data of the optimal strategy based on the strategy execution score.
[0071] As the system implementation is basically similar to the method implementation, the description is relatively simple, and relevant parts can be found in the description of the method implementation.
[0072] It is to be understood that the terms "first", "second", and the like, used in the description and in the claims, are used as adjectives to distinguish between similar objects, and do not necessarily have a specific order or sequence. It is to be understood that the data so used in the description and in the claims can be interchanged, as appropriate, to refer to a similar one of the other elements. The embodiments described in the following examples do not represent all the implementations consistent with the present application. Instead, they are merely examples consistent with some aspects of the present application.
[0073] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific feature or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative description of the above terms in the specification does not necessarily refer to the same embodiment or example. Moreover, the specific feature or characteristic described can be combined in any appropriate manner in one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in the specification.
[0074] Although the present application is described herein in conjunction with various embodiments, it is understood that other variations of the disclosed embodiments can be understood and realized by those skilled in the art upon review of the drawings and the disclosure. In the description of the present application, the word "comprising" does not exclude other components or steps, "a" or "one" does not exclude a plurality, and "multiple" means two or more, unless otherwise expressly specified. In addition, some measures are described in different embodiments, but this does not mean that these measures cannot be combined to produce good results.
[0075] The above is a further detailed description of the present application in conjunction with specific preferred embodiments, and the specific implementation of the present application cannot be limited to these descriptions. For those skilled in the art, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, which should be considered as falling within the scope of protection of the present application.
Claims
1. A method for generating strategies for complex scenarios based on decision trees and the UCB algorithm, characterized in that, include: Step 10: Obtain task scenario data; Step 20: Based on the task scenario data, generate an initial policy set using a pre-constructed decision tree; Step 30: Use the UCB algorithm to select the optimal strategy from the initial policy set. One strategy is used to obtain a set of feasible strategies; ≥1; Step 40: Based on the task scenario data, generate an adjustment plan for each strategy in the feasible strategy set, determine the auction price of the strategy based on the adjustment plan, and select the optimal strategy based on the auction price; the adjustment plan is used to map resources and objectives for the strategy; the auction price is used to comprehensively evaluate the capabilities of the adjustment plan in three dimensions: time cost, execution cost, and execution effect; Step 50: Execute the adjustment plan corresponding to the optimal strategy, calculate the strategy execution score based on the execution result, and update the historical performance data of the optimal strategy based on the strategy execution score.
2. The method for generating complex scene strategies based on decision trees and UCB algorithm according to claim 1, characterized in that, Step 30 specifically includes: Calculate the value of each policy in the initial policy set. value: ； in, Indicates the first One strategy value, Indicates the first The historical average task performance score of each strategy Indicates the potential score for exploration. This represents the preset exploration factor. Indicates the first The historical execution count of each strategy This represents the total number of historical executions of all strategies in the initial strategy set; Filter from the initial set of strategies The highest value One strategy is used to obtain a set of feasible strategies; the... The calculation method is as follows: ； in, This indicates the number of policies in the initial policy set.
3. The method for generating complex scene strategies based on decision trees and UCB algorithm according to claim 1, characterized in that, Step 40 specifically includes: Step 41: Based on the strategy type of each strategy in the feasible strategy set, generate an adjustment plan for each strategy through rule mapping or optimization algorithm, and monitor the generation time of the adjustment plan for each strategy. When the generation time exceeds the preset timeout threshold, terminate the generation of the adjustment plan for that strategy. Step 42: Calculate the auction price based on the adjustment plan: ； in, Indicates the index of the strategy corresponding to the adjustment plan. Indicates the first The auction price for each strategy, Indicates the first The time cost score for each strategy Indicates the first The execution cost score of each strategy. Indicates the first The score for the effectiveness of each strategy. , and These represent the time cost weight, execution cost weight, and execution effect weight, respectively. Step 43: Select the strategy with the highest auction price as the optimal strategy.
4. The method for generating complex scene strategies based on decision trees and UCB algorithms according to claim 3, characterized in that, The time cost score is calculated as follows: ； in, Indicates the first The time required to generate adjustment plans for each strategy. This represents the maximum time required to generate adjustment schemes for all strategies; The execution cost score is calculated as follows: ； in, , Indicates the first The implementation cost of adjusting the strategy. Indicates the first Each strategy adjustment plan utilizes a set of resources. Representing resources unit cost Representing resources The number of times it is used This represents the maximum execution cost of all strategy adjustment plans; The execution performance score is calculated as follows: ； in, Indicates the index of the evaluation dimension. Representing resources In the Scores for each evaluation dimension Indicates the first The weights of each evaluation dimension, Indicates the number of evaluation dimensions. Indicates the first The total number of resources used by each strategy adjustment plan.
5. The method for generating complex scene strategies based on decision trees and UCB algorithms according to claim 3, characterized in that, The method further includes: normalizing the auction price so that the strategy with the highest normalized auction price is the optimal strategy. The specific method for normalizing the auction price is as follows: ； in, This represents the normalized auction price.
6. The method for generating complex scene strategies based on decision tree and UCB algorithm according to claim 1, characterized in that, The step of calculating the strategy execution score based on the execution result and updating the historical performance data of the optimal strategy based on the strategy execution score includes: Obtain the task success rate and strategy execution time based on the execution results; Calculate the optimal strategy execution score based on the task success rate and strategy execution time: ； in, Indicates the strategy execution score. Indicates the task success rate. The weights representing the task success rate , Indicates the time factor. Indicates the strategy execution time. This indicates the preset baseline strategy time. Indicates multiplication operation; Update the historical execution count and historical average task performance score of the optimal strategy based on the strategy execution score.
7. The method for generating complex scene strategies based on decision trees and UCB algorithm according to claim 6, characterized in that, The update method for the historical execution count and historical average task performance score is as follows: ； ； in, This represents the historical average task performance score of the optimal strategy. This represents the historical execution count of the optimal strategy. This represents the updated historical average task performance score. This indicates the number of times the code has been executed in the past.
8. A complex scenario policy generation system based on decision tree and UCB algorithm, characterized in that, include: The acquisition module is used to acquire task scenario data; The strategy generation module is used to generate an initial strategy set based on the task scenario data using a pre-built decision tree. The strategy selection module is used to select the optimal strategy from the initial strategy set using the UCB algorithm. One strategy is used to obtain a set of feasible strategies; ≥1; The adjustment scheme generation and evaluation module is used to generate an adjustment scheme for each strategy in the feasible strategy set based on the task scenario data, determine the auction price of the strategy based on the adjustment scheme, and select the optimal strategy based on the auction price; the adjustment scheme is used to map resources and objectives for the strategy; the auction price is used to comprehensively evaluate the capabilities of the adjustment scheme in three dimensions: time cost, execution cost, and execution effect; The strategy evaluation and feedback module is used to execute the adjustment plan corresponding to the optimal strategy, calculate the strategy execution score based on the execution result, and update the historical performance data of the optimal strategy based on the strategy execution score.