A clustering-based unmanned system counter-patrol strategy generation and switching method

Abstract

The application discloses a clustering-based unmanned system counter-patrol strategy generation and switching method, and belongs to the field of intelligent decision of unmanned systems. An undirected patrol topology graph of a physical environment to be patrolled is acquired, and a target steady-state distribution is calculated. A patrol strategy uncertainty index composed of a strategy internal entropy and a strategy-to-strategy entropy weighted is defined, the strategy-to-strategy entropy takes an average absolute difference between state transition matrices as a distance, a condensed hierarchical clustering and an average linkage method are used to divide a strategy set into a plurality of clusters, and a Shannon entropy is calculated according to a probability distribution of a scale proportion of each cluster. A double-entropy balance optimization algorithm is used to generate a strategy set satisfying constraints through a steady-state constraint nearest point projection, a disturbance type transition matrix generator and an iterative search of a simulated annealing main process. A two-level switching protocol aligned with a clustering structure is used to control an unmanned system to perform a patrol task. The problems of inconsistent dimensions of existing index, incomplete diversity description and single switching mode are solved, and the unpredictability of a patrol path is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent decision-making technology for unmanned systems, and in particular to a clustering-based method for generating and switching anti-patrol strategies for unmanned systems. Background Technology

[0002] With the rapid development of unmanned systems technology, autonomous unmanned systems, represented by drones, unmanned vehicles, and unmanned surface vessels, have been widely used in tasks such as area patrol and target surveillance due to their advantages of high mobility, strong persistence, and low manpower costs. In adversarial environments, patrollers need to move between multiple patrol positions according to certain strategies, while attackers may estimate their behavioral patterns by continuously observing the patroller's historical trajectory, and thus predict future paths. To improve the unpredictability of patrol strategies, existing technologies typically use Markov chain models to describe the transition probabilities of patrollers between different positions, and increase the difficulty of prediction for attackers by generating multiple state transition matrices and switching them in a certain way. However, how to uniformly quantify the unpredictability of the patrol strategy set, how to effectively characterize the diversity of the strategy set, and how to structurally switch multiple state transition matrices during deployment remain problems that need further solutions.

[0003] In existing technologies, the unpredictability of patrol strategies is typically calculated by a weighted average of intra-strategy entropy and inter-strategy entropy, with inter-strategy entropy often measured by the arithmetic mean of the average absolute differences between matrices. This measurement method is inconsistent with the physical meaning of intra-strategy entropy and does not comprehensively characterize the diversity of the strategy set. Furthermore, during the deployment phase, patrollers usually switch between different state transition matrices using a uniform random sampling method. This relatively simple switching method makes it difficult to continuously increase the difficulty for attackers to estimate patrol patterns from historical trajectories. These problems mean that existing patrol strategies still face the risk of being gradually estimated and predicted when facing attackers with continuous observation capabilities. Summary of the Invention

[0004] This invention provides a clustering-based method for generating and switching anti-patrol strategies for unmanned systems, aiming to solve the problems of inconsistent unpredictability measurements, incomplete characterization of strategy set diversity, and the simplistic strategy switching methods that make it easy for attackers to predict patrol patterns. The embodiments of this invention provide a clustering-based method for generating and switching anti-patrol strategies for unmanned systems, which includes: Obtain the undirected patrol topology graph of the physical environment to be patrolled, and calculate the steady-state distribution of the target based on the value parameters and attack time parameters of each node in the undirected patrol topology graph; A patrol strategy uncertainty index is defined, which is a weighted average of intra-strategy entropy and inter-strategy entropy, used to evaluate a strategy set consisting of multiple state transition matrices. Intra-strategy entropy is the arithmetic mean of the steady-state weighted entropy of each state transition matrix in the strategy set, and steady-state weighted entropy is the weighted average of the row entropy of each state transition matrix with the target steady-state distribution as the weight. Inter-strategy entropy is calculated by using the average absolute difference between the state transition matrices in the strategy set as the distance, and using agglomerative hierarchical clustering and average linkage methods to divide the strategy set into several clusters according to a predetermined clustering threshold. The Shannon entropy is calculated by using the proportion of the number of state transition matrices contained in each cluster to the total number of strategy sets as the probability distribution. Both intra-strategy entropy and inter-strategy entropy are Shannon entropy in bits. A clustering-based dual-entropy balance optimization algorithm is adopted to maximize the patrol strategy uncertainty index as the optimization objective. Under the conditions of satisfying topological constraints, row randomness constraints, element non-negativity constraints, and target steady-state distribution constraints, iterative search is performed through steady-state constraint nearest point projection, perturbation-type transition matrix generator, and simulated annealing main process to generate a strategy set containing multiple state transition matrices that satisfy the constraints. The unmanned control system executes patrol missions based on the generated policy set, and adopts the same clustering structure as when calculating the entropy between policies for a two-level switching protocol: the first level is inter-cluster switching, which arranges all clusters in random order so that the state transition matrices used before and after come from different clusters; the second level is intra-cluster sampling, which randomly and uniformly samples a state transition matrix from the current cluster, executes a random number of transition steps, and then switches to the next cluster.

[0005] The technical solution provided by this invention calculates the steady-state distribution of targets based on node value and attack time, enabling high-value nodes with short attack times to obtain a higher access probability, thereby ensuring consistency between patrol priorities and protection needs at the resource allocation level. It constructs intra-strategy entropy using the target steady-state distribution as weight, and calculates inter-strategy entropy on the cluster size distribution after dividing the clusters using the average absolute difference between state transition matrices as distance and agglomerative hierarchical clustering and average linking method. Both are Shannon entropies in bits, thus uniformly characterizing the unpredictability of the strategy set from both micro-randomness and macro-diversity levels, overcoming the shortcomings of existing metrics such as inconsistent dimensions and difficulty in identifying grouping structures. By aiming to maximize this index, it addresses topological constraints, row randomness constraints, element non-negativity constraints, and target steady-state... Under distribution constraints, a strategy set is generated through steady-state constraint nearest point projection, perturbation-type transition matrix generator, and simulated annealing main process iterative search. This ensures that the generated strategy set satisfies the constraints while possessing both internal randomness within a single matrix and differences between multiple matrices. A two-level switching protocol aligned with the clustering structure is used to control the unmanned system to perform patrol tasks. Inter-cluster switching ensures that the state transition matrices used before and after are from different clusters, causing the attacker's observation sequence to continuously alternate between transition patterns with significant structural differences. Intra-cluster extraction ensures that the specific matrix within the same cluster is unpredictable. Both of these factors together inject continuous non-stationarity, making it difficult for attackers to converge to any single state transition matrix estimate based on the frequency statistics of historical trajectories. This increases the attacker's prediction error, enhances the unpredictability of the patrol path, and strengthens the protection effect. Attached Figure Description

[0006] Figure 1 A flowchart illustrating a clustering-based method for generating and switching anti-patrol strategies for unmanned systems, as provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a Markov chain containing five nodes in an embodiment of the present invention. Figure 3 This is a trend graph showing the change in the number of clusters obtained by clustering as a function of the clustering threshold in an embodiment of the present invention. Figure 4 This is a trend graph showing the change of inter-strategy entropy with clustering threshold in an embodiment of the present invention; Figure 5 This is a trend graph showing the change of strategy entropy with clustering threshold in an embodiment of the present invention; Figure 6 This is a trend graph showing the change of the patrol strategy uncertainty index with the clustering threshold in an embodiment of the present invention; Figure 7 This is a trend graph showing the change of the average relative error of key states with the number of observation steps under different clustering thresholds in this embodiment of the invention. Figure 8This is a trend graph showing the change of importance-weighted average row error with the number of observation steps under different clustering thresholds in this embodiment of the invention. Figure 9 This is a trend graph showing the change of the average relative error of the critical state with the number of observation steps under three algorithm configurations in this embodiment of the invention; Figure 10 This is a trend graph showing the change of importance-weighted average row error with the number of observation steps under three algorithm configurations in this embodiment of the invention; Figure 11 This is a graph showing the percentage improvement in the average relative error of the two configurations relative to the baseline configuration in the embodiments of the present invention. Figure 12 This is a weighted average row error improvement percentage chart for the two configurations relative to the baseline configuration in this embodiment of the invention. Detailed Implementation

[0007] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0008] The overall concept of this invention is as follows: For unmanned system patrol missions in adversarial environments, the physical environment to be patrolled is first abstracted into an undirected patrol topology graph, and the steady-state distribution of the target is calculated based on the value parameters and attack time parameters of each node. Based on this, the patrol strategy uncertainty index is reconstructed and decomposed into two components: intra-strategy entropy and inter-strategy entropy. The intra-strategy entropy is the arithmetic mean of the steady-state weighted entropies of each state transition matrix in the strategy set, and the inter-strategy entropy is the distance between the average absolute differences between the state transition matrices. Agglomerated hierarchical clustering and average linking methods are used to divide the strategy set into several clusters according to a predetermined clustering threshold, and Shannon entropy is calculated on the size distribution of each cluster, so that both components are Shannon entropy in bits. Subsequently, a dual-entropy balance optimization algorithm based on clustering patterns is adopted, with the optimization objective of maximizing the patrol strategy uncertainty index. Through a three-layer nested collaborative search using steady-state constraint nearest point projection, a perturbation-type transition matrix generator, and a simulated annealing main process, a strategy set is generated under the conditions of satisfying topological constraints, row randomness constraints, element non-negativity constraints, and target steady-state distribution constraints. Finally, a two-level switching protocol is executed to control the unmanned system to perform patrol tasks using the same clustering structure as when calculating inter-strategy entropy. The first level is inter-cluster switching, and the second level is intra-cluster extraction, ensuring that the state transition matrices used before and after the transition come from different clusters and that intra-cluster extraction is unpredictable. Furthermore, an attribution verification system is constructed to separate the contributions of the strategy generation algorithm and the switching protocol. Therefore, this invention provides a systematic solution in terms of unpredictability index dimensional consistency, strategy set grouping structure identification, structured strategy switching, and separable contribution verification.

[0009] The embodiments of the present invention will now be described in detail with reference to specific steps, such as... Figure 1 As shown in the figure, the method for generating and switching adversarial patrol strategies for unmanned systems based on clustering provided by this invention may include the following steps: S110: Obtain the undirected patrol topology map of the physical environment to be patrolled, and calculate the steady-state distribution of the target based on the value parameters and attack time parameters of each node in the undirected patrol topology map.

[0010] Specifically, the adversarial patrol scenario involved in this embodiment includes three types of objects: environment, patroller, and attacker. The environment model, patroller model, and attacker model are established below.

[0011] Environment Model: The physical environment to be patrolled is discretized and modeled using an undirected topological graph. ,in, The set representing vertices (patrol positions), each node in the graph This represents locations in the real-world environment that require patrolling and surveillance. This represents the total number of patrol positions. The set representing the edges, Represents the node Transfer to node The probability satisfies .like Figure 2 Let represent a Markov chain with five nodes, whose corresponding state transition matrix is ​​irreducible and aperiodic. The transition probabilities between nodes are encoded in the state transition matrix. The unmanned system can start from any initial vertex and then sample the probability distribution in the row corresponding to its current vertex to determine which area to patrol next.

[0012] Patrol Model: The unmanned system moves on the Markov chain model according to the state transition matrix. The set of transition matrices is divided into several clusters through clustering. The unmanned system switches matrices at two levels: between clusters and within clusters. This makes it difficult for attackers to accurately estimate the transition matrix from historical trajectories, effectively reducing the risk of patrol patterns being predicted.

[0013] Attacker Model: It is assumed that the attacker has global observation capabilities, can continuously record the historical location sequence of the unmanned system, and estimate its state transition matrix accordingly.

[0014] Different nodes Different nodes represent different locations that need patrolling, and each node has different value. Attackers need different amounts of time to attack different nodes. Therefore, high-value target nodes and nodes with short attack times require more patrol resources, and unmanned systems are more likely to access these nodes. The steady-state distribution of targets is calculated based on the value parameters and attack time parameters of each node. The target steady-state distribution represents the expected probability distribution of the unmanned system visiting each node during long-term patrols, and satisfies the probability distribution normalization condition. The target steady-state distribution provides a resource allocation strategy based on target importance and vulnerability, ensuring that the patrol strategy prioritizes the protection of targets that are "high-value and have short attack times."

[0015] S120 defines a patrol strategy uncertainty index, which is a weighted average of intra-strategy entropy and inter-strategy entropy, used to evaluate a set of strategies consisting of multiple state transition matrices. Intra-strategy entropy is the arithmetic mean of the steady-state weighted entropies of each state transition matrix in the strategy set. The steady-state weighted entropy is the weighted average of the row entropies of each state transition matrix, weighted by the target steady-state distribution. Inter-strategy entropy uses the average absolute difference between the state transition matrices in the strategy set as the distance. Agglomerated hierarchical clustering and average linkage methods are used to divide the strategy set into several clusters according to a predetermined clustering threshold. The Shannon entropy is calculated using the proportion of state transition matrices contained in each cluster to the total number of strategies in the set as the probability distribution. Both intra-strategy entropy and inter-strategy entropy are Shannon entropies expressed in bits.

[0016] Specifically, let the set of strategies be... ,in, The number of state transition matrices in the policy set. For the first There are several state transition matrices. The policy entropy measures the randomness of transitions within a single state transition matrix: For each state transition matrix in the policy set, calculate the entropy of each row, where the row entropy is the Shannon entropy of the transition probability distribution of that row; using the target steady-state distribution as the weight, sum the row entropies to obtain the steady-state weighted entropy of the state transition matrix; calculate the arithmetic mean of the steady-state weighted entropies of all state transition matrices in the policy set to obtain the policy entropy.

[0017] Inter-policy entropy is used to measure the diversity of differences between different state transition matrices: the average of the absolute differences between corresponding elements of two state transition matrices is used as the policy distance; using the policy distance as a metric, agglomerative hierarchical clustering and average linking methods are used to divide the policy set into several clusters according to a predetermined clustering threshold; the number of state transition matrices contained in each cluster is counted, and the proportion of the number of state transition matrices contained in each cluster to the total number of policy sets is calculated; Shannon entropy is calculated using the proportion of each cluster as a probability distribution to obtain the inter-policy entropy.

[0018] The patrol strategy uncertainty index is constructed by weighting intra-strategy entropy and inter-strategy entropy, where the weighting coefficients of both intra-strategy entropy and inter-strategy entropy are greater than zero. Both intra-strategy and inter-strategy entropy are Shannon entropy expressed in bits, ensuring consistent dimensions and preventing any imbalance in magnitude during weighted summation. This patrol strategy uncertainty index is used to evaluate a set of strategies comprised of multiple state transition matrices; a higher score indicates greater unpredictability of the strategy set.

[0019] S130 employs a clustering-based dual-entropy balance optimization algorithm. With the goal of maximizing the patrol strategy uncertainty index, it generates a strategy set containing multiple state transition matrices that satisfy the constraints by performing iterative searches through steady-state constraint nearest point projection, perturbation-type transition matrix generator, and simulated annealing main process, while satisfying topological constraints, row randomness constraints, element non-negativity constraints, and target steady-state distribution constraints.

[0020] Specifically, given the adjacency matrix A of the patrol area and the steady-state distribution of the target... and the number of matrices The optimization problem is formalized as follows:

[0021] Among them, the topology constraint requires that the unmanned system can only move along the edges that exist in the undirected patrol topology graph; the element non-negativity constraint requires that each element in the state transition matrix is ​​non-negative; the row randomness constraint requires that the sum of the elements in each row of each state transition matrix is ​​1, ensuring that each row constitutes a legal probability distribution; the target steady-state distribution constraint requires that the actual steady-state distribution corresponding to the generated policy set is consistent with the target steady-state distribution.

[0022] The optimization problem described above has the following characteristics, making traditional gradient methods unsuitable for direct application: First, the inter-policy entropy involves hierarchical clustering operations, and the output of clustering is a discrete cluster partition with a specific number of clusters. First, since cluster membership is not differentiable with respect to matrix elements, the inter-policy entropy is also not differentiable with respect to decision variables. Second, the objective steady-state distribution constraint is a column constraint coupled across rows, for a given matrix... Topological constraints, element nonnegativity constraints, and row randomness constraints can be satisfied independently row by row, but the objective steady-state distribution constraint connects all rows together. The steady-state constraints of the column involve the first row of all rows. Because there are only a few elements, the candidate matrix generated row by row generally cannot satisfy all four types of constraints at the same time; third, the objective function couples information at the set level and the matrix level. The entropy within a policy can be decomposed into the independent contribution of each matrix, but the entropy between policies depends on the pairwise distance matrix and clustering structure of the entire set, and cannot be decomposed into independent subproblems of a single matrix.

[0023] Based on the above analysis, this embodiment adopts simulated annealing as the optimization framework. Simulated annealing does not rely on the gradient information of the objective function, but only needs to evaluate the function values ​​of candidate solutions. It is naturally suitable for optimization problems containing non-differentiable operators. At the same time, it accepts inferior solutions with positive probability through the Metropolis acceptance criterion, and has the ability to escape local optima. It is suitable for handling non-convex problems with uncertainty exponents in patrol strategies.

[0024] The clustering-based dual-entropy balance optimization algorithm employs a three-layer nested structure: the simulated annealing main process resides in the outermost layer, managing the entire search process; the perturbation-type transition matrix generator resides in the middle layer, responsible for generating candidate transition matrices on top of the base matrices; and the steady-state constraint nearest point projection resides in the innermost layer, responsible for projecting the candidate matrices onto the feasible region to precisely satisfy the four types of constraints mentioned above. During the initialization phase, the simulated annealing main process generates multiple independent random base matrices and calls the perturbation-type transition matrix generator to generate the initial state transition matrix, calculating the patrol policy uncertainty index of the current policy set as a baseline value. In the iterative search phase, each round randomly selects an index, and the perturbation-type transition matrix generator regenerates the state transition matrix corresponding to that index with random parameters, while the remaining state transition matrices remain unchanged, forming a candidate policy set. The patrol policy uncertainty index of the candidate policy set is calculated, and the acceptance of the candidate policy set is determined based on the simulated annealing acceptance criterion. After each round, the temperature decays according to geometric cooling scheduling. A set-level rebalancing is performed every set number of rounds to maintain the balance between the two target dimensions of intra-policy entropy and inter-policy entropy. Through the above iterative search, a policy set containing multiple state transition matrices that satisfy the constraints is finally generated.

[0025] S140: The unmanned control system executes patrol missions based on the generated policy set, employing the same clustering structure as when calculating inter-policy entropy for a two-level switching protocol: The first level is inter-cluster switching, where all clusters are arranged in random order so that the state transition matrices used before and after are from different clusters. The second level is intra-cluster sampling, where a state transition matrix is ​​randomly and uniformly sampled from the current cluster, and after executing a random number of transition steps, the system switches to the next cluster.

[0026] Specifically, after obtaining the policy set and its clustering structure, the policy set is deployed to the unmanned system performing patrol missions. During the patrol process, the unmanned system extracts the state transition matrix from the policy set according to a two-level switching protocol.

[0027] The first level is inter-cluster switching: the clusters obtained from clustering are arranged in a random order, and the unmanned system visits each cluster in turn. When the unmanned system visits a certain cluster, the state transition matrices used before and after are from different clusters, thus ensuring that the state transition matrices used successively have significant differences in the transition rules.

[0028] The second level is in-cluster sampling: When the unmanned system visits a cluster, a state transition matrix is ​​randomly and uniformly sampled from that cluster. A random number of transition steps are then executed according to this matrix. After execution, the system switches to the next cluster and repeats the process. Once all clusters have been visited in one round, a new random order is generated, and the next round of traversal begins. If all state transition matrices within a cluster have been used, that cluster is refilled and its order is shuffled to ensure that subsequent sampling remains random.

[0029] Through the aforementioned two-level switching protocol, unmanned systems exhibit high unpredictability on both short-term and long-term scales, making it difficult for attackers to estimate the patroller's state transition matrix from historical trajectories, thereby improving the protection effect on critical infrastructure.

[0030] The technical solution provided by this invention calculates the steady-state distribution of targets based on node value and attack time, enabling high-value nodes with short attack times to obtain a higher access probability, thereby ensuring consistency between patrol priorities and protection needs at the resource allocation level. It constructs intra-strategy entropy using the target steady-state distribution as weight, and calculates inter-strategy entropy on the cluster size distribution after dividing the clusters using the average absolute difference between state transition matrices as distance and agglomerative hierarchical clustering and average linking method. Both are Shannon entropies in bits, thus uniformly characterizing the unpredictability of the strategy set from both micro-randomness and macro-diversity levels, overcoming the shortcomings of existing metrics such as inconsistent dimensions and difficulty in identifying grouping structures. By aiming to maximize this index, it addresses topological constraints, row randomness constraints, element non-negativity constraints, and target steady-state... Under distribution constraints, a strategy set is generated through steady-state constraint nearest point projection, perturbation-type transition matrix generator, and simulated annealing main process iterative search. This ensures that the generated strategy set satisfies the constraints while possessing both internal randomness within a single matrix and differences between multiple matrices. A two-level switching protocol aligned with the clustering structure is used to control the unmanned system to perform patrol tasks. Inter-cluster switching ensures that the state transition matrices used before and after are from different clusters, causing the attacker's observation sequence to continuously alternate between transition patterns with significant structural differences. Intra-cluster extraction ensures that the specific matrix within the same cluster is unpredictable. Both of these factors together inject continuous non-stationarity, making it difficult for attackers to converge to any single state transition matrix estimate based on the frequency statistics of historical trajectories. This increases the attacker's prediction error, enhances the unpredictability of the patrol path, and strengthens the protection effect.

[0031] exist Figure 1 Based on the illustrated embodiment, as one implementation of this invention, calculating the steady-state distribution of the target based on the value parameters and attack time parameters of each node in the undirected patrol topology graph may include the following steps: Step a1: Obtain the value parameters and attack time parameters of each node in the undirected patrol topology graph.

[0032] Specifically, for undirected patrol topology graphs Each node in To obtain its value parameters and attack time parameters Among them, value parameters Representation Nodes The importance of a facility in the overall defense mission can be pre-set based on factors such as its strategic significance and asset value; attack timing parameters. Indicates that the attacker successfully attacked the node. The required time reflects the vulnerability of the node. These parameters can be configured according to the specific needs of the actual application scenario.

[0033] Step a2: Calculate the importance weight parameter of each node based on the value parameter and attack time parameter of each node. The importance weight parameter is equal to the ratio of the node's value parameter to its attack time parameter.

[0034] Specifically, for each node Calculate its importance weight parameters : ; This weighting parameter comprehensively reflects the relationship between a node's value and the difficulty of attacking it: the higher the value, the shorter the time required for the attack. The larger the value, the higher the priority that the node should receive in the allocation of patrol resources.

[0035] Step a3: Normalize the importance weight parameters of each node to obtain the target steady-state distribution probability value of each node. The target steady-state distribution probability value represents the expected probability of the unmanned system visiting the corresponding node during long-term patrol.

[0036] Specifically, for each node Its target steady-state distribution probability value The calculation formula is: ; in, Let be the total number of nodes in the topology graph. The denominator represents the sum of importance weights for all nodes, used to normalize the numerator so that the sum of the steady-state probability distributions of all nodes is 1. The steady-state probability distribution of the targets characterizes the expected probability distribution of the unmanned system visiting each node during long-term patrols. This distribution provides a basis for resource allocation in the generation of subsequent patrol strategies, ensuring that high-value nodes with short attack times receive higher patrol access frequencies.

[0037] The technical solution provided by this invention obtains the value parameters and attack time parameters of each node, calculates the importance weight parameters and normalizes them to obtain the target steady-state distribution probability value. This allows the target steady-state distribution to comprehensively reflect the value of the node and the attack time, thereby ensuring that the patrol focus and protection requirements are consistent at the resource allocation level. This enables high-value nodes with short attack times to obtain a higher access probability, providing a reasonable resource allocation basis for the subsequent generation of patrol strategies that meet the target steady-state distribution constraints. This helps to prioritize the protection of key nodes and improve the overall protection effect.

[0038] exist Figure 1Based on the illustrated embodiment, as one implementation of this invention, the calculation of the policy internal entropy may include the following steps: Step b1: For each state transition matrix in the policy set, calculate the Shannon entropy of the transition probability distribution of each row as the row entropy.

[0039] Specifically, let the set of strategies be... ,in L The number of state transition matrices in the policy set. For the first A state transition matrix, It is the first l From node j in the transition matrix To the node The transition probability, n Let be the total number of nodes. For each state transition matrix in the policy set... Calculate its first i row entropy : ; in, Representation matrix The i Line, line entropy ) indicates that the unmanned system starts from the node The uncertainty in the probability distribution of transitioning from a starting node to other nodes. A higher row entropy indicates a greater likelihood of transitioning from a node... The more dispersed the starting transfer directions are, the more difficult they are to predict; conversely, if the transfer probability is concentrated on a few nodes, the row entropy value is smaller, and the transfer behavior is more deterministic.

[0040] Step b2: Using the target steady-state distribution as the weight, the row entropy is weighted and summed to obtain the steady-state weighted entropy of the state transition matrix.

[0041] Specifically, for the state transition matrix Using the target steady-state distribution as the weight, the row entropy of each row is weighted and summed to obtain the matrix. steady-state weighted entropy : ; Among them, the target steady-state distribution This reflects the node activity of unmanned systems during long-term patrols. Expected access frequency. A weighted average is applied to the weights, ensuring that nodes with higher visit frequencies during actual patrols contribute more significantly to the overall uncertainty measure, thus more accurately reflecting the degree of randomness in the actual execution of the patrol strategy. Steady-state weighted entropy. Integrating matrices The uncertainty information of all rows is used to characterize the overall uncertainty of a single state transition matrix.

[0042] Step b3: Calculate the arithmetic mean of the steady-state weighted entropies of all state transition matrices in the policy set to obtain the policy internal entropy.

[0043] Specifically, for the set of strategies Steady-state weighted entropy of all state transition matrices Calculate the arithmetic mean to obtain the policy internal entropy. : ; in, The number of state transition matrices in the policy set. Policy internal entropy. It reflects the average level of randomness of all state transition matrices in the policy set. The higher the value, the more difficult it is for an attacker to predict the single-step transition behavior of the unmanned system within a single state transition matrix.

[0044] The technical solution provided by this invention quantifies the microscopic randomness within a single state transition matrix into a single numerical index through the aforementioned three-layer progressive calculation: sequentially calculating row entropy, steady-state weighted entropy, and ensemble arithmetic mean. Using information entropy as a measure of row entropy objectively reflects the uniformity of the transition probability distribution originating from each node, with sufficient theoretical basis. Introducing the target steady-state distribution as a weighting factor ensures that the contribution of different nodes to the overall randomness measurement remains consistent with the actual patrol resource allocation strategy, making the evaluation results more aligned with the actual patrol mission requirements. By calculating the arithmetic mean of all matrices in the strategy set, the potential random bias of a single matrix is ​​eliminated, enabling the strategy entropy to stably characterize the overall microscopic unpredictability level of the strategy set, providing a reliable quantitative basis for the horizontal comparison and optimization of patrol strategies.

[0045] exist Figure 1 Based on the illustrated embodiment, as one implementation of this invention, the calculation of inter-strategy entropy may include the following steps: Step c1: Use the average of the absolute differences between the corresponding elements of the two state transition matrices as the policy distance.

[0046] Specifically, let the set of strategies be... ,in L The number of state transition matrices in the policy set. and Let be any two distinct state transition matrices in the policy set. Representation matrix The Middle Line 1 The elements of the column, i.e., from the node Transfer to node The probability, n Total number of nodes. Policy distance. Defined as the average of the absolute differences between corresponding elements of two state transition matrices: ; The policy distance measures the average deviation of two state transition matrices across all state transition probabilities. A larger distance indicates a more significant difference in the transition structure between the two policies.

[0047] Step c2: Using policy distance as a metric, treat each state transition matrix in the policy set as an independent cluster and construct an initial distance matrix.

[0048] Specifically, the set of strategies Each state transition matrix in Treat it as an independent cluster ,common Initial clusters. Initial distance matrix. It consists of the policy distance between each matrix pair: ; in, Represents the first position in the initial distance matrix. Line 1 The elements of the column, i.e., the state transition matrix and The strategic distance between them.

[0049] Step c3: Repeat the following steps until the distance between all clusters exceeds the predetermined clustering threshold. Find the two clusters with the smallest current distance. If the current distance does not exceed a predetermined clustering threshold... If the two clusters are not found to be merged into a new cluster, the cluster set is updated to remove the two merged clusters and add them to the new cluster. The average linking method is then used to calculate the distance between the new cluster and the remaining clusters to update the distance matrix. The average linking method calculates the average policy distance between all pairs of state transition matrices in the two clusters.

[0050] Specifically, in the first In the iterative process, find the two clusters with the smallest distance. and : ; in, This represents the number of iteration steps. and These are the indices of the two clusters with the smallest current distance.

[0051] If the minimum distance does not exceed the predetermined clustering threshold Then the two clusters will be merged into a new cluster: ; Update the cluster set, remove the two merged clusters and add the new cluster: ; The distance between the new cluster and the other clusters is calculated using the average linkage method: ; in, Cluster The number of state transition matrices included. The average linking method measures the overall distance between two clusters by calculating the average policy distance between all pairs of state transition matrices in the two clusters.

[0052] Step c4: When the distance between all clusters exceeds the predetermined clustering threshold, terminate the clustering and obtain several clusters.

[0053] Specifically, when the distance between all cluster pairs exceeds a predetermined clustering threshold At that time, that is: ; The algorithm terminates, and finally obtains Cluster Each state transition matrix is ​​assigned to a unique cluster. Clustering threshold. Controlling cluster granularity: smaller More clusters are generated, and a larger θ produces fewer clusters.

[0054] Step c5: Count the number of state transition matrices in each cluster and calculate the proportion of the number of state transition matrices in each cluster to the total number of policies.

[0055] Specifically, statistics for each cluster Number of state transition matrices included : ; in, For indicator functions, when Belongs to cluster The value is 1 if the condition is met, and 0 otherwise. The size of each cluster satisfies the following conditions: Calculate the proportion of each cluster. : ; This ratio indicates that when a state transition matrix is ​​randomly selected, the matrix belongs to the [number]th [order]. The probability of each cluster. When all clusters are of equal size. For all When a cluster is established, the distribution is most uniform; when a cluster contains the vast majority of matrices, the distribution is highly concentrated, and the diversity of the strategy set is low.

[0056] Step c6: Calculate the Shannon entropy using the proportion of each cluster as the probability distribution, and obtain the inter-strategy entropy: .

[0057] in, This is the clustering threshold. The number of clusters obtained from clustering. For the first The proportion of the number of state transition matrices contained in a cluster to the total number of policies.

[0058] Inter-policy entropy measures the dispersion of the set of state transition matrices across different clusters. It reaches its maximum when all matrices are uniformly distributed across the clusters, and zero when all matrices are concentrated in the same cluster. Inter-policy entropy satisfies the following mathematical property: non-negativity, i.e. The equality holds if and only if Upper bound, i.e. The equality holds if and only if each state transition matrix forms a self-contained family; monotonicity, i.e. Regarding clustering thresholds Monotonous and does not increase.

[0059] The technical solution provided by this invention divides the policy set into several clusters using agglomerative hierarchical clustering and average linking methods, with the average absolute difference between state transition matrices as the policy distance. Shannon entropy is calculated on the size distribution of each cluster, making the inter-policy entropy Shannon entropy in the information theory sense, sharing the same bit-based dimension as the intra-policy entropy, thus avoiding any imbalance in magnitude during weighted summation. Simultaneously, the inter-policy entropy, constructed based on cluster distribution, possesses the ability to identify the grouping structure of the policy set, distinguishing between multiple significantly different policy clusters and numerous small variants of the same policy, overcoming the shortcomings of existing element-wise difference measures that cannot identify grouping structures. Furthermore, the inter-policy entropy satisfies non-negativity, upper bound, and monotonically non-increasing property with respect to the clustering threshold, providing a theoretical basis for selecting the clustering threshold and a reliable macroscopic diversity measure for constructing and subsequently optimizing the patrol policy uncertainty index.

[0060] exist Figure 1 Based on the illustrated embodiment, as one implementation of this invention, the logical policy uncertainty index is composed of a weighted average of intra-policy entropy and inter-policy entropy, and its calculation formula is as follows: ; in, For the internal entropy of the strategy, For inter-strategy entropy, For a predetermined clustering threshold, The weighting coefficients are the internal entropy of the strategy. Let be the weighting coefficient of the inter-strategy entropy, and , The weighting coefficients are all greater than zero, indicating that both the intra-strategy entropy and inter-strategy entropy components participate in the composition of the patrol strategy uncertainty index, and neither can be omitted.

[0061] Inter-strategy entropy The specific calculation formula and symbol meaning based on cluster distribution are as described in step c6, and will not be repeated here.

[0062] The technical solution provided by this invention ensures that the patrol strategy uncertainty index takes into account both the micro-randomness within a single state transition matrix and the macro-diversity between different state transition matrices by limiting the weight coefficients of both intra-strategy entropy and inter-strategy entropy to be greater than zero. This avoids the loss of evaluation dimensions due to a component having a weight of zero. Furthermore, the non-negativity, upper bound, and monotonicity of inter-strategy entropy provide a theoretical basis for selecting the clustering threshold. This allows the determination of whether the clustering threshold is too large or too small to be determined by the change in inter-strategy entropy, thus providing a quantitative basis for determining the optimal clustering threshold through adversarial verification. This helps improve the accuracy of the patrol strategy uncertainty index in assessing the unpredictability of the strategy set.

[0063] exist Figure 1 Based on the illustrated embodiment, as one implementation of this invention, steady-state constraint nearest point projection is used to correct the candidate matrix obtained by the perturbation-type transition matrix generator, and may include the following steps: Step d1: Extract the elements of the candidate matrix into candidate vectors from the allowed edge set. The allowed edge set is the set of node pairs with edges in the undirected patrol topology graph.

[0064] Specifically, let's define an undirected patrol topology graph. The adjacency matrix is The allowed set of edges is Let be the set of nodes in an undirected patrolling topology where edges exist, and denote the allowed number of edges as . The candidate matrix output by the perturbation-type transition matrix generator is denoted as... Candidate matrix In the set of allowed edges The elements on the vector are extracted as candidate vectors in a preset order. ∈ ,Right now ,in express The topological constraints are satisfied by projecting the elements on the allowed edges into a real vector space. Since the elements of the candidate matrix at the non-allowed edge positions must be zero, the topological constraints can be satisfied simply by projecting the elements on the allowed edges and keeping the elements on the non-allowed edges zero.

[0065] Step d2: Under the conditions of satisfying row randomness constraints, objective steady-state distribution constraints, and element non-negativity constraints, construct a quadratic programming problem that minimizes the Euclidean distance between feasible vectors and candidate vectors.

[0066] Specifically, the matrix projection problem is transformed into a vector projection problem on allowed edges. Let the feasible vectors to be found be... , This corresponds to the elements of the transition matrix satisfying all constraints on the set of allowed edges. The projection problem is modeled as the following quadratic programming problem: ; Wherein, objective function feasible vector With candidate vectors Minimize the squared Euclidean distance between the candidate vectors, which is the feasible vector with the smallest Euclidean distance to the candidate vector. Equality constraints. It combines row randomness constraints and target steady-state distribution constraints: the row randomness constraint requires that the sum of the elements in each row of each state transition matrix be 1, for a total of n Item; Target steady-state distribution constraint requirement: For all ,have Because there is a linear correlation between the row randomness constraint and the target steady-state distribution constraint, the actual retention... Randomness constraints and A total of linearly independent target steady-state distribution constraints. Linear equality constraints. Inequality constraints. The corresponding element non-negativity constraint requires that all elements on allowed edges in the transition matrix be non-negative. (Constraint matrix in the projection problem) sum vector Depends solely on the target steady-state distribution Adjacency matrix of the undirected patrol topology graph , and candidate matrix Irrelevant.

[0067] Step d3 involves using sequential quadratic programming to solve the quadratic programming problem and obtain the optimal feasible vector.

[0068] Specifically, the sequential quadratic programming algorithm is used to solve the above quadratic programming problem to obtain the optimal feasible vector. : ; Since the objective function is a convex quadratic function and the constraints are linear equality and linear inequality, this quadratic programming problem is a convex quadratic programming problem, and the sequential quadratic programming problem can converge to the global optimum. The optimal feasible vector obtained from the solution is... That is, the distance candidate vector on the allowed edge set. The closest feasible vector that satisfies all constraints. To ensure numerical stability, the solution result can be truncated to non-negativity, i.e. .

[0069] Step d4: Map the optimal feasible vector back to matrix form to obtain the state transition matrix that satisfies the topological constraints, row randomness constraints, and target steady-state distribution constraints.

[0070] Specifically, the optimal feasible vector The inverse operation of extracting elements in step d1 is performed to map them back to matrix form, resulting in the corrected state transition matrix. : ; in, This indicates that the set of edges will be allowed. The element vector on is restored to n × n The matrix mapping sets the elements at non-allowed edge positions to zero. Because... The matrix satisfies row randomness constraints, objective steady-state distribution constraints, and element non-negativity constraints, and the elements of non-allowed edges are zero. Therefore, the corrected matrix... It precisely satisfies topological constraints, row randomness constraints, element nonnegativity constraints, and the target steady-state distribution constraints. Furthermore, since projection is the feasible correction that minimizes the altered Euclidean norm, therefore... While satisfying all constraints, it retains the entropy and diversity characteristics bestowed by the perturbation-type transition matrix generator to the greatest extent possible.

[0071] Step d5: The constraint matrix and vectors of the quadratic programming problem depend only on the target steady-state distribution and the undirected patrol topology graph. They are pre-constructed once during the initialization phase and shared in subsequent projection calls.

[0072] Specifically, the constraint matrix in the projection problem sum vector Only by the target steady-state distribution Adjacency matrix of undirected patrol topology graph It was decided that the constraint matrix would only need to be pre-constructed once during the algorithm initialization phase. Thereafter, in each iteration of the simulated annealing main process, the perturbation-type transition matrix generator would share the same set of constraint matrices each time it called the steady-state constraint nearest point projection. sum vector This eliminates the need for repeated construction. Consequently, it avoids repeatedly calculating the constraint matrix and vectors during each projection, significantly reducing computational overhead and improving the overall efficiency of the algorithm.

[0073] The technical solution provided by this invention extracts the elements of the candidate matrix on the allowed edge set as candidate vectors, constructs a quadratic programming problem that minimizes the Euclidean distance between the feasible vector and the candidate vector, and solves it using sequential quadratic programming. This allows the candidate matrix output by the perturbation-type transition matrix generator to be accurately projected onto the feasible region, simultaneously satisfying topological constraints, row randomness constraints, element non-negativity constraints, and target steady-state distribution constraints. Since projection is the feasible correction with minimal modification, it preserves the entropy and diversity characteristics given by the generator to the maximum extent, avoiding the destruction of the randomness and diversity of the strategy set due to forced correction. At the same time, the use of general steady-state constraints rather than detailed balance conditions allows for searching over a wider feasible region, which is beneficial for finding a better strategy set. Furthermore, the constraint matrix and vector are pre-constructed once during the initialization phase and shared in subsequent projection calls, avoiding repeated calculations and improving algorithm efficiency. This provides stable and efficient projection support for the iterative search of the clustering-based dual-entropy balance optimization algorithm.

[0074] As one implementation of this invention, a perturbation-type transition matrix generator is used to generate candidate transition matrices on top of a base matrix. The inputs of the perturbation-type transition matrix generator include the base matrix, mixing coefficients, diversity factor, and entropy focusing flag. The base matrix is ​​an initial transition matrix that satisfies topological constraints and row randomness constraints. The perturbation-type transition matrix generator includes: Step e1: Process each row of the base matrix as the current row in turn: Determine the set of allowed columns for the current row, which is the set of nodes in the undirected patrol topology graph that have edges with the corresponding node of the current row; Construct normalized random noise on the set of allowed columns, where the element position of the corresponding node of the current row that moves to itself is the self-loop position, and the self-loop position is set to zero; The current row is the current value of the corresponding row of the base matrix after topological masking, and the current row and normalized random noise are linearly mixed according to the mixing coefficient. The linear mixing is as follows: The current row is weighted by the value after subtracting the mixing coefficient from 1, the normalized random noise is weighted by the mixing coefficient, and then the weighted current row and the weighted normalized random noise are summed. The mixing coefficient controls the degree to which the current row deviates from the base matrix.

[0075] Specifically, let the basis matrix be... The adjacency matrix of the undirected patrol topology is The mixing coefficient is Diversity factor is Entropy focusing is marked by First, consider the basis matrix. Perform topological masking to obtain the mask matrix. , where ⊙ denotes element-wise multiplication, meaning only elements in the basis matrix corresponding to the positions of existing edges in the undirected patrol topology are retained, while elements at non-allowed edge positions are set to zero. Then, the mask matrix... Each line is processed sequentially as the current line. For the first... The row determines its set of allowed columns, i.e., all columns that satisfy the condition. The column The set formed represents the node corresponding to the current row. The set of nodes with edges.

[0076] Construct normalized random noise on the allowed column set For each column in the allowed column set except for self-loop positions: Generate uniform random numbers Set the self-loop position to zero, i.e. =0; set the non-allowed column positions to zero. Then normalize the noise vector to obtain normalized random noise: after normalization, all elements of the noise vector ξ are non-negative and their sum is 1.

[0077] According to the mixing coefficient Linearly mix the current row with normalized random noise: ; in, For mask matrix The The row is the current value of the corresponding row after the basis matrix is ​​masked by the topological mask; The mixing coefficient controls the degree to which the current row deviates from the base matrix. The larger the value, the more the candidate rows deviate from the basis matrix, thus increasing the difference between matrices; The smaller the value, the closer the candidate row is to the basis matrix, thus preserving a higher single-row entropy.

[0078] Step e2: When the entropy focusing flag is enabled, the self-loop probability of the corresponding node in the current row of the base matrix is ​​weighted with the random number according to the diversity factor to determine the self-loop probability of the current row. The self-loop position is assigned the self-loop probability, and the non-self-loop elements in the current row except for the self-loop position are scaled. The scaling is to multiply the non-self-loop elements by the scaling factor, which is equal to 1 minus the self-loop probability and then divided by the sum of the non-self-loop elements, so that the sum of the elements in the current row is 1.

[0079] Specifically, when When enabled, based on the diversity factor Determine the probability of a self-loop in the current row. : ; in, basis matrix The Middle Line 1 The elements of the column represent the self-loop probabilities of the nodes corresponding to the current row in the basis matrix; Uniformly random numbers, ; It serves as a diversity factor, controlling the random amplitude of the self-loop probability. The larger the value, the closer the self-loop probability is to a random number; The smaller the value, the closer the self-loop probability is to the self-loop probability of the basis matrix.

[0080] Then calculate the sum of all non-self-looping elements in the current row. : ; If s > 0, then the non-self-looping elements in the current row, excluding self-looping positions, are scaled by a scaling factor of 0. ,Right now: · ; At the same time, the self-loop position is assigned the self-loop probability: ; in The sum of the current non-self-looping elements is 1. After the above processing, the sum of all elements in the current row is 1.

[0081] Step e3: When the entropy focusing flag is not enabled, set the self-loop probability of the current row to 1 minus the sum of all non-self-loop elements in the current row.

[0082] Specifically, when When not enabled, entropy equalization is not performed; the self-loop probability of the current row is directly set to: ; This method determines the probability value of a self-loop position by using row sums and padding, ensuring that the sum of all elements in the current row is 1. This approach is computationally simple and suitable for scenarios where additional row entropy increase is not required.

[0083] Step e4: Normalize the current row to eliminate numerical errors.

[0084] Specifically, the current row after processing in step e2 or step e3 is normalized, and then the mask matrix is ​​updated. The The current line after behavior normalization: This step is used to eliminate numerical errors that may be introduced by floating-point operations, ensuring that the sum of the elements in each row is exactly 1.

[0085] Step e5: After processing row by row, the nearest point projection of steady-state constraints is called to project the matrix onto the feasible region, and the state transition matrix that satisfies all constraints is output.

[0086] Specifically, when the mask matrix After all rows have been processed, the steady-state constraint nearest point projection is called to transform the matrix. The projection is applied to the feasible region to precisely satisfy the topological constraints, row randomness constraints, element nonnegativity constraints, and target steady-state distribution constraints. The projected matrix is ​​the candidate transition matrix output by the perturbation-type transition matrix generator.

[0087] The technical solution provided in this invention injects parameterized random perturbations row by row into the base matrix using a perturbation-type transition matrix generator. A mixing coefficient controls the perturbation intensity, a diversity factor controls the random amplitude of self-loop probabilities, and an entropy focusing flag determines whether entropy equalization is performed. This generates candidate transition matrices that combine randomness and diversity while maintaining topological constraints and row randomness. Linear mixing allows candidate rows to be flexibly adjusted between the base matrix and random noise, preserving the feasible structure of the base matrix while introducing sufficient randomness. Weighting of self-loop probabilities and scaling of non-self-loop elements when the entropy focusing flag is enabled increases the entropy of a single row, preventing excessive concentration of candidate matrices. Row-by-row normalization and projection of the nearest point of steady-state constraints ensure that the output matrix accurately satisfies all constraints. This provides high-quality, diverse candidate transition matrices for iterative search in a clustering-based dual-entropy balance optimization algorithm, helping to improve the unpredictability of the strategy set.

[0088] Based on the above embodiments, as one implementation of the present invention, the simulated annealing main process includes: Step f1, Initialization Phase: Set the initial temperature T 0, generate L The n independent random basis matrices, for the nth l Given a random basis matrix, an initial state transition matrix is ​​generated by calling a perturbation-type transition matrix generator using an increasing diversity factor and alternating entropy focusing flags. The diversity factor is equal to 0.2 plus... l / L The product of 0.6, where L The number of state transition matrices in the policy set, when l When the entropy focusing flag is even, it is enabled. l When the entropy focus flag is odd, it is disabled; the patrol policy uncertainty index of the current policy set is calculated as the baseline value.

[0089] Specifically, set the initial temperature T 0. Termination temperature T end Cooling rate and the maximum number of iterations .generate independent random basis matrices Each random basis matrix independently samples uniform random numbers from the allowed column set and normalizes them into a row random matrix.

[0090] For the random basis matrices Using an increasing diversity factor and alternating entropy focusing signs Call the perturbation transition matrix generator to generate the initial state transition matrix. :

[0091] ; When calling the perturbation-type transition matrix generator, the mixing coefficient is set to 0.4. The index of the random basis matrix. ; This represents the number of state transition matrices in the policy set. (Diversity factor) With index Incrementing, with values ​​ranging from 0.2 to 0.8, ensures that each matrix in the initial policy set has varying degrees of self-loop randomness; entropy focusing flag. With index Alternate between enabling and disabling, when When it is even To enable, when When it is an odd number If not enabled, some matrices in the initial policy set undergo entropy equalization, while others retain their base matrix structure, thus giving the initial set sufficient diversity.

[0092] After initialization, calculate the patrol policy uncertainty index of the current policy set. As a benchmark value: ; And initialize the set of historical best strategies. and the corresponding historical best value : , .

[0093] Step f2, iterative search phase: In each round, an index is randomly selected, and the perturbation transition matrix generator is called with random parameters to regenerate the state transition matrix corresponding to that index. The remaining state transition matrices remain unchanged, forming a candidate policy set.

[0094] Specifically, in the it-th iteration, an index is randomly selected. Call the perturbation-type transition matrix generator with random parameters: ; ;

[0095] in, The mixing coefficient, As a diversity factor, As an entropy-focusing indicator, the perturbation-type transition matrix generator is denoted as... , The first generated in step f1 The nth random basis matrix. The perturbation-type transition matrix generator is invoked to regenerate the nth random basis matrix. State transition matrix : ; The remaining state transition matrices remain unchanged, forming a candidate policy set. .

[0096] Step f3: Calculate the patrol policy uncertainty index of the candidate policy set, and decide whether to accept the candidate policy set according to the Metropolis acceptance criterion. The Metropolis acceptance criterion is: if the patrol policy uncertainty index of the candidate policy set is greater than that of the current policy set, it is unconditionally accepted; otherwise, it is accepted with probability. Accept, among which, The difference between the patrol policy uncertainty index of the candidate policy set and the patrol policy uncertainty index of the current policy set is given. The current temperature is set; if accepted, the current policy set is updated to the candidate policy set, and if the uncertainty index of the updated patrol policy is better than the historical best value, the current policy set is recorded as the historical best policy set.

[0097] Specifically, the candidate policy set is calculated. Patrol strategy uncertainty index : ; Calculate the increment of the objective function : ; in, The difference between the patrol policy uncertainty index of the candidate policy set and the patrol policy uncertainty index of the current policy set is used. The decision on whether to accept the candidate policy set is made according to the Metropolis acceptance criterion. ; in, The current temperature. Since the objective is to maximize the patrol strategy uncertainty index, when... Accept directly at the time; when Time with probability Accept. Temperature The higher the temperature T, the greater the probability of accepting inferior solutions, which is beneficial for escaping local optima; the lower the temperature T, the more the search tends to be greedy, gradually converging to a high-quality solution.

[0098] If the candidate policy set is accepted, the current policy set will be updated to the candidate policy set: ; ; When the updated patrol strategy uncertainty index is better than the historical best value, the current strategy set is recorded as the historical best strategy set: like ,but ; ; Step f4: After each round, the temperature decays according to the geometric cooling schedule.

[0099] Specifically, after each iteration, the temperature decays according to geometric cooling schedule: ; in, The temperature gradually decreases with increasing iteration number, and the search behavior gradually transitions from early global exploration to later local convergence.

[0100] Step f5 involves performing a set-level rebalancing every set number of rounds to maintain the balance between the two target dimensions: intra-strategy entropy and inter-strategy entropy. In this embodiment, the set number of rounds can be 50, meaning a set-level rebalancing is performed every 50 rounds.

[0101] Specifically, the process checks whether the policy entropy of each state transition matrix in the current policy set is too low and whether the state transition matrices are too similar. For state transition matrices with low policy entropy, the perturbation-type transition matrix generator is called again to generate new state transition matrices to replace the original matrices. For state transition matrices that are too similar to each other, the parameters of at least one of the state transition matrices are adjusted and regenerated to increase the difference between the matrices. After the rebalancing is completed, the patrol policy uncertainty index of the current policy set is recalculated, and the historical best policy set is updated if it is better than the historical best value.

[0102] The technical solution provided in this invention drives a global search by maximizing the patrol strategy uncertainty index through simulated annealing as the optimization objective. It generates an initial strategy set using an increasing diversity factor and alternating entropy focusing flags, endowing the initial set with sufficient diversity. Candidate state transition matrices are regenerated by calling a perturbation-type transition matrix generator with random parameters, forming a candidate strategy set while keeping the other matrices unchanged, allowing the search process to specifically improve individual matrices. The Metropolis acceptance criterion accepts inferior solutions with positive probability, enabling the algorithm to escape local optima and making it suitable for handling non-convex problems related to the patrol strategy uncertainty index. Geometric cooling scheduling gradually reduces the temperature, allowing the search behavior to smoothly transition from global exploration to local convergence. By performing set-level rebalancing every set number of rounds, it actively repairs matrices with excessively low intra-strategy entropy or overly similar matrices, explicitly maintaining the balance between the two objective dimensions of intra-strategy entropy and inter-strategy entropy, preventing the algorithm from converging to a single-dimensional local optimum, thereby generating a high-quality, highly unpredictable strategy set.

[0103] Based on the above embodiments, as one implementation of the present invention, it further includes determining a clustering threshold, specifically: Neither extreme of the clustering threshold provides optimal defense. When the clustering threshold is too small, almost every state transition matrix forms its own cluster, the number of clusters approaches the total number of policies, and the two-level switching degenerates into a uniform traversal of all state transition matrices, with the cluster structure no longer providing additional diversity organization. When the clustering threshold is too large, all state transition matrices are merged into a very small number of clusters, the number of clusters approaches one, the range of choices for switching between clusters is extremely limited, the entropy between policies approaches zero, and the structural differences of the policy set are significantly compressed. Therefore, the effective clustering granularity lies between the two extremes, and the optimal value needs to be determined through parameter scanning and adversarial verification.

[0104] Specifically, parameter scanning refers to generating corresponding policy sets for multiple candidate clustering thresholds and evaluating their defensive effectiveness; adversarial verification refers to simulating an attacker estimating the state transition matrix based on the observed trajectory, using the attacker's prediction error as the criterion for evaluating the defensive effectiveness. Steps g1 and g2 are explained in detail below.

[0105] Step g1: For multiple candidate clustering thresholds, generate corresponding policy sets respectively. Deploy the policy sets generated under each candidate clustering threshold according to the two-level switching protocol and simulate patrol trajectories. The attacker estimates the state transition matrix of the patroller based on the observed trajectory using a transition frequency maximum likelihood estimator.

[0106] Specifically, multiple candidate clustering thresholds are pre-set. For each candidate clustering threshold, a dual-entropy balance optimization algorithm based on clustering patterns is used to generate the corresponding policy set under that threshold. The generated policy set is deployed in the unmanned system according to a two-level switching protocol to simulate the unmanned system performing patrol tasks. During the simulated patrol, the unmanned system extracts a state transition matrix from the policy set according to the two-level switching protocol: the first level is inter-cluster switching, where the clusters are arranged in a random order, and the unmanned system visits each cluster sequentially; the second level is intra-cluster extraction, where a state transition matrix is ​​randomly and uniformly extracted from the current cluster, and a transition step of random length is executed according to the state transition matrix before switching to the next cluster. The number of transition steps executed after each extraction of the state transition matrix can be randomly and uniformly extracted from the interval [100, 300] until the cumulative number of steps reaches the preset time span. This generates the patrol state sequence of the unmanned system.

[0107] The attacker has global observation capabilities and can continuously record the historical position sequence of the unmanned system. Without any prior knowledge, the attacker can only observe the state transition sequence of the unmanned system and use a maximum likelihood estimator of the transition frequency to estimate the state transition matrix of the unmanned system.

[0108] Step g2 uses the average relative error of critical states and the importance-weighted average row error as evaluation indicators to determine the optimal clustering threshold that maximizes the attacker's prediction error.

[0109] Specifically, for each candidate clustering threshold, the corresponding average relative error of critical states and importance-weighted average row error are calculated. The larger the average relative error of critical states and the importance-weighted average row error, the greater the attacker's estimation bias of the unmanned system's state transition matrix; that is, the stronger the resilience of the strategy set generated under that candidate clustering threshold against attacker predictions. The candidate clustering threshold that maximizes the attacker's prediction error is selected as the optimal clustering threshold.

[0110] Step g3: The transition frequency maximum likelihood estimator is a transition probability estimate constructed based on the frequency of occurrence of each state pair in the observed state transition sequence. The key state average relative error is the arithmetic mean of the relative errors between the corresponding rows of the actual state transition matrix and the estimated state transition matrix on a preset set of key nodes. The importance-weighted average row error is a weighted average of the transition probability estimation errors of each row, weighted by the target steady-state distribution.

[0111] Specifically, the calculation methods for the maximum likelihood estimator of the transition frequency, the average relative error of the critical state, and the importance-weighted average row error can be referred to the relevant existing technologies, and will not be elaborated on in this embodiment.

[0112] The technical solution provided by this invention generates strategy sets for multiple candidate clustering thresholds and deploys simulated patrol trajectories according to a two-level switching protocol. This enables attackers to estimate the state transition matrix based on observed trajectories, simulating their estimation behavior in adversarial scenarios. By using the average relative error of key states and the importance-weighted average row error as evaluation indicators, the optimal clustering threshold that maximizes the attacker's prediction error is determined, avoiding the bias that may result from determining the threshold solely based on the patrol strategy uncertainty index. Thus, the optimal clustering threshold determined through adversarial verification can provide the best defense effect for unmanned systems in actual adversarial scenarios, maximizing the attacker's prediction error based on historical trajectories and significantly improving the unpredictability and protective effectiveness of the patrol strategy.

[0113] Based on the above embodiments, as one implementation of the present invention, it further includes constructing an attribution verification system, specifically: Step h1: Set the baseline configuration, use the Metropolis-Hastings algorithm to generate a policy set and uniformly and randomly sample the state transition matrix; set the dual-entropy balance optimization configuration, use a clustering-based dual-entropy balance optimization algorithm to generate a policy set and uniformly and randomly sample the state transition matrix; set the dual-entropy balance clustering switching configuration, use a clustering-based dual-entropy balance optimization algorithm to generate a policy set and use a two-level switching protocol to sample the state transition matrix.

[0114] Specifically, to evaluate the respective contributions of the strategy generation algorithm and the switching protocol, this embodiment sets up three algorithm configurations, all sharing the same execution flow: state transition matrices are sequentially extracted from the strategy set. The unmanned system executes a random number of transition steps for each state transition matrix before switching to the next, until the cumulative number of steps reaches a preset time span. The number of transition steps executed after each extraction of a state transition matrix can be uniformly and randomly selected from the interval [100, 300]. The differences between the three configurations lie only in the strategy set generation algorithm and the extraction method of the state transition matrices.

[0115] The baseline configuration uses the Metropolis-Hastings algorithm to generate the policy set, and the state transition matrix is ​​extracted using a uniform random sampling method.

[0116] The dual-entropy balance optimization configuration uses a clustering-based dual-entropy balance optimization algorithm to generate the policy set, while the state transition matrix is ​​still extracted using a uniform random sampling method. The only difference between the dual-entropy balance optimization configuration and the baseline configuration lies in the algorithm for generating the policy set; therefore, the performance difference between the two reflects solely the contribution of the clustering-based dual-entropy balance optimization algorithm at the policy generation level.

[0117] The dual-entropy balanced clustering switching configuration uses a dual-entropy balanced optimization algorithm based on clustering patterns to generate a set of strategies, and the extraction of the state transition matrix is ​​replaced by a two-level switching protocol. The only difference between the dual-entropy balanced clustering switching configuration and the dual-entropy balanced optimization configuration is the method of extracting the state transition matrix; therefore, the performance difference between the two reflects the contribution of the two-level switching protocol alone.

[0118] Step h2: Run the baseline configuration, dual-entropy balance optimization configuration, and dual-entropy balance clustering switching configuration respectively to generate the corresponding patrol trajectory; use the transition frequency maximum likelihood estimator to estimate the state transition matrix based on the patrol trajectory, and calculate the average relative error of key states and the importance-weighted average row error.

[0119] Specifically, the three configurations described above are run separately, each generating a corresponding patrol trajectory. The attacker uses a transition frequency maximum likelihood estimator to estimate the state transition matrix based on the observed patrol trajectories and calculates the average relative error of critical states and the importance-weighted average row error. The specific calculation method for the evaluation metrics is the same as in step g3, and will not be repeated here. Each configuration can be run under multiple sets of different random seeds to obtain stable evaluation results.

[0120] Step h3 involves measuring the improvement of the attack prediction error difference between the baseline configuration and the dual-entropy balance optimized configuration, the improvement of the attack prediction error difference switching protocol between the dual-entropy balance optimized configuration and the dual-entropy balance clustering switching configuration, and the total improvement of the attack prediction error difference between the baseline configuration and the dual-entropy balance clustering switching configuration. The total improvement is the sum of the improvements in the strategy generation algorithm and the switching protocol. The total improvement is used as the evaluation basis to guide the parameter adjustment of the dual-entropy balance optimization algorithm based on the clustering pattern and the deployment and verification of the two-level switching protocol.

[0121] Specifically, the three configurations described above constitute a clear attribution chain: the performance difference measurement from the baseline configuration to the dual-entropy balanced optimized configuration is based on the improvement of the strategy generation algorithm; the performance difference measurement from the dual-entropy balanced optimized configuration to the dual-entropy balanced clustering switching configuration is based on the improvement of the switching protocol; and the total performance difference measurement from the baseline configuration to the dual-entropy balanced clustering switching configuration is the sum of the improvements in the strategy generation algorithm and the switching protocol. Through this attribution chain, the contributions of the strategy generation algorithm and the switching protocol can be separated and quantified, thus providing a clear quantitative basis for subsequent targeted algorithm optimization and engineering deployment.

[0122] The technical solution provided by this invention establishes a progressively layered comparison structure by setting three algorithm configurations: baseline configuration, dual-entropy balance optimization configuration, and dual-entropy balance clustering switching configuration. This allows the contributions of the strategy generation algorithm and the switching protocol to be separated and quantified. By measuring the performance differences between configurations using the difference in attacker prediction error, the defensive gains of the dual-entropy balance optimization algorithm based on clustering patterns and the two-level switching protocol can be objectively evaluated. By using the total improvement as the evaluation criterion to guide parameter adjustment and deployment verification, a clear quantitative basis is provided for subsequent targeted algorithm optimization and engineering deployment, which helps to further improve the unpredictability and protection effectiveness of patrol strategies.

[0123] To verify the effectiveness of the clustering-based unmanned system adversarial patrol strategy generation and switching method provided in the above embodiments, this embodiment conducts system verification through simulation experiments. Specifically, this includes: firstly, setting a clustering threshold... Perform parameter scanning to analyze the patrol strategy uncertainty index and its two components, as well as the cluster number. The study examines the changing trends of clustering thresholds; secondly, it analyzes the impact of clustering thresholds on defense effectiveness and determines the optimal clustering threshold; finally, it uses the maximum likelihood estimator of transition frequency as the attacker model for verification, evaluates the defense gain of the clustering pattern-based dual-entropy balance optimization algorithm and the two-level switching protocol relative to the baseline configuration, and separates the contributions of the generation algorithm and the switching protocol along the attribution chain separation strategy.

[0124] like Figures 3 to 6 As shown, the clustering threshold A parameter scan was performed within the range [0.05, 0.15], and the number of matrices... Take 10 levels from 10 to 100. Figure 3 This shows the number of clusters obtained from the clustering. (Right now Figure 3 The number of clusters on the y-axis varies with the clustering threshold. The changing trend Figure 4 It shows the entropy between strategies With clustering threshold The changing trend Figure 5 It shows the internal entropy of the strategy With clustering threshold The changing trend Figure 6 Displaying the patrol strategy uncertainty index (Right now Figure 3 The total uncertainty of the ordinate varies with the clustering threshold. The changing trend. Different colored curves in each graph correspond to different numbers of matrices. The experimental results are as follows.

[0125] like Figure 3 As shown, the number of clusters With clustering threshold The value increases and then monotonically decreases, consistent with the aforementioned property that the entropy between strategies does not monotonically increase with respect to the clustering threshold. When At that time, most state transition matrices form a family of their own. The corresponding number of clusters is 82; when When the value is increased to 0.14–0.15, the number of clusters in all configurations decreases to 2, and the grouping granularity of the policy set tends to be coarser.

[0126] like Figure 4 As shown, inter-strategy entropy The number of clusters shows a decreasing trend that is highly consistent with the number of clusters. When When the size is small, the state transition matrix is ​​distributed across a large number of clusters, and the cluster sizes are nearly uniform. The value is relatively high; with As the number of clusters increases, the number of clusters decreases, and the size of each cluster tends to be uneven. Significant decrease. Number of matrices. The larger, in the same The more matrices available for allocation, The higher the absolute value, the better.

[0127] like Figure 5 As shown, the internal entropy of the strategy Clustering threshold The value is not sensitive to changes and fluctuates within a narrow range of 1.60–1.80 across all configurations. This indicates that the choice of clustering threshold primarily affects the grouping structure of the policy set without altering the level of randomness within individual state transition matrices, consistent with the design intent of measuring different levels of unpredictability between intra-policy entropy and inter-policy entropy.

[0128] like Figure 6 As shown, the patrol strategy uncertainty index The trend is determined by the superposition of two components: intra-strategy entropy and inter-strategy entropy. Since intra-strategy entropy is approximately constant, The changes are mainly driven by inter-strategy entropy, and the overall change is driven by inter-strategy entropy. It decreases as it increases. However, Maximum does not directly equate to optimal defensive effect. When the size is small, the number of clusters approaches the number of matrices. The two-level switching protocol degenerates into a uniform traversal, and the clustering structure loses its ability to organize diversity. Therefore, the determination of the optimal clustering threshold cannot be based solely on... The value needs to be determined in conjunction with subsequent adversarial verification, with the attacker's prediction error as the final criterion.

[0129] The aforementioned clustering threshold sensitivity analysis shows that the patrol strategy uncertainty index monotonically decreases as the clustering threshold increases, but the value of this index is not directly equivalent to the defense effect. To determine the optimal clustering threshold, this embodiment, from the perspective of the unmanned system, simulates the attacker's estimation process, uses the attacker's prediction error as the criterion for evaluating the defense effect, and selects the clustering threshold that maximizes the prediction error as the optimal value. Referring to the settings of relevant prior art, the unmanned system selects 50 state transition matrices in the strategy set, and deploys the strategy set according to a two-level switching protocol. The number of transition steps executed after each extraction of the state transition matrix is ​​uniformly and randomly extracted from the interval [100, 300].

[0130] Assuming the attacker has no prior knowledge and can only observe the state transition sequence of the unmanned system, a maximum likelihood estimator based on transition frequency is used to estimate the state transition matrix. Evaluation metrics include the mean relative error of critical states and the importance-weighted average row error.

[0131] Figure 7 and Figure 8 The trends of attacker prediction error with the number of observation steps are shown for different clustering thresholds. Figure 7 This corresponds to the average relative error of critical states (KS-ARE). Figure 8 This corresponds to the importance-weighted average row error (IW-ARE). Each curve corresponds to a candidate clustering threshold, and the shaded area represents the 95% confidence interval for 100 different random seeds. Figure 7 and Figure 8 The following patterns can be observed: First, the prediction errors corresponding to all clustering thresholds rise rapidly when the number of observation steps is small, and then tend to stabilize, indicating that the non-stationarity injected by the two-level switching protocol effectively prevents the attacker's estimation convergence. Second, the curve corresponding to a clustering threshold of 0.10 is consistently at the highest level on both evaluation metrics, outperforming other clustering thresholds for most observation steps, indicating that the cluster partitioning granularity at this threshold is most beneficial for defense. The performance of a clustering threshold of 0.08 is similar, constituting a suboptimal choice. Third, the prediction error at a clustering threshold of 0.05 is relatively low among all candidate clustering thresholds, verifying the aforementioned analysis. Too small a clustering threshold leads to too many clusters, causing the two-level switching protocol to degenerate into a uniform traversal, and the cluster structure fails to effectively organize diversity. The prediction errors at clustering thresholds of 0.14 and 0.15 are also relatively low, corresponding to the degenerate situation where the number of clusters is too small and the inter-strategy entropy is close to zero.

[0132] The above results indicate that the optimal clustering threshold lies between two extremes and is inconsistent with the monotonically decreasing trend of the patrol strategy uncertainty index: the clustering threshold of 0.05, with the highest patrol strategy uncertainty index, did not yield the optimal defense effect, while the clustering threshold of 0.10, with a patrol strategy uncertainty index in the middle, performed best in adversarial testing. This result suggests that the determination of the optimal clustering threshold must be evaluated through adversarial experiments, rather than solely based on the numerical value of the index. Therefore, subsequent experiments uniformly adopted 0.10 as the optimal clustering threshold.

[0133] Furthermore, to evaluate the respective contributions of the policy generation algorithm and the switching protocol, this embodiment sets up three algorithm configurations, forming a progressively layered comparison structure. The three configurations share the same execution flow: state transition matrices are sequentially extracted from the policy set; the unmanned system executes a random number of transition steps for each state transition matrix before switching to the next, until the cumulative number of steps reaches a preset time span. The differences between the three configurations lie only in the policy set generation algorithm and the extraction method of the state transition matrices. The experimental setup is consistent with the optimal clustering threshold determination section. The attacker uses a maximum likelihood estimator for the transition frequency, and the number of transition steps executed each time is uniformly and randomly extracted from the interval [100, 300]. Each configuration is run under 100 different random seeds, and the number of observation steps ranges from 200 to 2000 across 10 levels.

[0134] The baseline configuration (MH) uses the Metropolis-Hastings algorithm to generate the policy set, and the state transition matrix is ​​extracted using a uniform random sampling method.

[0135] The dual-entropy balance optimization configuration (CM-DEBO) uses a clustering-based dual-entropy balance optimization algorithm (CM-DEBO algorithm) to generate the policy set, while the state transition matrix is ​​still extracted using a uniform random sampling method. The only difference between the dual-entropy balance optimization configuration and the baseline configuration lies in the algorithm for generating the policy set; therefore, the performance difference between the two reflects solely the contribution of the clustering-based dual-entropy balance optimization algorithm at the policy generation level.

[0136] The Dual Entropy Balanced Clustering Switching Configuration (CM-DEBO-clu) uses the Dual Entropy Balanced Optimization Algorithm (CM-DEBO) based on clustering patterns to generate a policy set, and replaces the extraction of the state transition matrix with a two-level switching protocol. The only difference between the Dual Entropy Balanced Clustering Switching Configuration (CM-DEBO-clu) and the Dual Entropy Balanced Optimization Configuration (CM-DEBO) lies in the extraction method of the state transition matrix; therefore, the performance difference between the two reflects the contribution of the two-level switching protocol alone.

[0137] The above design constitutes a clear attribution chain: the improvement of the baseline configuration (MH) to dual-entropy balanced optimized configuration (CM-DEBO) metric strategy generation algorithm, the improvement of the dual-entropy balanced optimized configuration (CM-DEBO) to dual-entropy balanced clustering switching configuration (CM-DEBO-clu) metric switching protocol, and the baseline configuration (MH) to dual-entropy balanced clustering switching configuration (CM-DEBO-clu) is the total improvement of the two combined.

[0138] Figure 9 and Figure 10 The trends of attacker prediction error with the number of observation steps are shown for three different algorithm configurations. Figure 9 The corresponding critical state mean relative error (KS-ARE) Figure 10 Corresponding to the importance-weighted average row error (IW-ARE). The shaded area represents the 95% confidence intervals for 100 different random seeds. From Figure 9 and Figure 10 Consistent hierarchical separation can be observed: the prediction error curve of CM-DEBO-clu is at the highest level across all observation steps, CM-DEBO is in the middle, and MH is the lowest. All three curves rise rapidly in the initial observation phase and then stabilize, without showing a continuous decline with increasing observation steps, indicating that the non-stationarity injected by the two-level switching protocol effectively prevents the attacker's estimation from converging.

[0139] Specifically, in terms of the critical state mean relative error index, CM-DEBO-clu reaches approximately 0.53 at 800 observation steps, CM-DEBO reaches approximately 0.44, while MH reaches only 0.33. In terms of the importance-weighted average row error index, CM-DEBO-clu reaches approximately 0.56 at 800 observation steps, CM-DEBO reaches approximately 0.46, and MH reaches approximately 0.37. The ranking of the three configurations remains stable across all observation steps.

[0140] Figure 11 and Figure 12 The percentage improvement in error relative to the MH baseline is shown for both CM-DEBO configurations. Figure 11 The corresponding critical state mean relative error (KS-ARE) Figure 12 The corresponding importance-weighted average row error (IW-ARE) is used. The error bar represents the bootstrap 95% confidence interval. Analysis along the attribution chain allows us to separate the contributions of the policy generation algorithm and the switching protocol.

[0141] The contribution of the policy generation algorithm is reflected in the improvement of CM-DEBO uniform sampling relative to the MH baseline. This improvement is approximately 20.5%–38.0% on KS-ARE and approximately 9.8%–29.8% on IW-ARE, indicating that the policy set generated by CM-DEBO itself has stronger anti-prediction capabilities than MH.

[0142] The contribution of the two-level handover protocol is reflected in the incremental improvement of CM-DEBO-clu relative to the uniform sampling of CM-DEBO. On KS-ARE, the total improvement of CM-DEBO-clu relative to the MH baseline is approximately 26.4%–60.6%, and after deducting the contribution of the policy generation algorithm, the two-level handover protocol contributes an additional 0.6%–25.4%. On IW-ARE, the total improvement is approximately 22.4%–53.6%, and the two-level handover protocol contributes an additional 3.7%–29.4%. This increment is positive across all observation steps, indicating that the two-level handover protocol provides a stable additional defense gain on top of the policy generation algorithm.

[0143] The above results verify the effectiveness of the attribution chain. The contributions of the policy generation algorithm and the two-level switching protocol are separable and superimposed. CM-DEBO-clu, as the complete method in this embodiment, significantly outperforms the MH baseline across all observation steps in both evaluation metrics.

[0144] This embodiment addresses the problem of unpredictability measurement and optimization of time-varying Markov policy sets in adversarial patrol scenarios, proposing a systematic solution. The main conclusions are as follows: (1) A patrol strategy uncertainty index composed of intra-strategy entropy and inter-strategy entropy is proposed. The two components measure the randomness within a single matrix and the diversity across matrices, respectively. The inter-strategy entropy is constructed based on agglomerative hierarchical clustering and can distinguish structural diversity.

[0145] (2) A dual-entropy balance optimization algorithm (CM-DEBO) based on clustering pattern was designed, which directly uses the patrol strategy uncertainty index as the optimization target. The algorithm adopts the simulated annealing framework and ensures that the output of each step satisfies the topological constraints, row randomness constraints, element non-negativity constraints and target steady-state distribution constraints through the steady-state constraint nearest point projection.

[0146] (3) A two-level patrol strategy switching protocol aligned with the clustering structure was designed. The inter-cluster switching ensures that the state transition matrices used before and after are from different clusters, and the random sampling within the cluster ensures that the selected state transition matrices within the same cluster are unpredictable. Together, they inject continuous non-stationarity into the attacker's observation sequence.

[0147] (4) Simulation experiments verified the effectiveness of the proposed method: clustering threshold sensitivity analysis showed that the optimal clustering threshold was located between two extremes and needed to be determined through adversarial verification; attribution analysis confirmed that the contributions of the strategy generation algorithm and the two-level switching protocol were separable and superimposed.

[0148] The technical solutions provided in the embodiments of the present invention have the following beneficial effects: (1) The inter-policy entropy is reconstructed. The average absolute difference between state transition matrices is used as the distance. The policy set is divided into several clusters through agglomerative hierarchical clustering, and Shannon entropy is calculated on the size distribution of each cluster. Unlike the prior art which directly uses the average absolute difference as the inter-policy entropy, the inter-policy entropy of this invention is Shannon entropy in the information theory sense. It is measured in bits and has the same dimension as the intra-policy entropy. There is no imbalance in magnitude when the two components are weighted and summed.

[0149] (2) Inter-strategy entropy has the ability to identify the grouping structure of strategy sets. Through clustering operations, it can distinguish between two fundamentally different diversity structures: "multiple significantly different strategy clusters" and "a large number of small variations of the same strategy". Existing element-wise difference-based measurement methods give similar values ​​for these two cases and cannot distinguish them.

[0150] (3) The entropy between strategies satisfies strict mathematical properties: non-negativity, that is, it is zero when all state transition matrices belong to the same cluster; upper bound, that is, it is the maximum value when each state transition matrix forms its own cluster; and monotonically non-increasing with respect to the clustering threshold, which provides a theoretical basis for the selection of the clustering threshold.

[0151] (4) A steady-state constraint nearest point projection operator was designed. The candidate matrix is ​​projected to the nearest point in the feasible region through quadratic programming. General steady-state constraints are used instead of detailed equilibrium conditions to search in a wider feasible region. The projection is the feasible correction with the least modification, which preserves the entropy and diversity characteristics given by the perturbation-type transition matrix generator to the maximum extent.

[0152] (5) A three-layer nested CM-DEBO algorithm was designed, which directly uses the patrol policy uncertainty index as the optimization target. Since the inter-policy entropy includes non-differentiable clustering operations, the algorithm uses a simulated annealing framework for global search and actively repairs the state transition matrix with excessively low entropy within the policy and the state transition matrix that is too similar to each other through a set-level rebalancing mechanism, thus explicitly maintaining the balance of the two target dimensions.

[0153] (6) A two-level patrol strategy switching protocol aligned with the clustering structure was designed. Inter-cluster switching ensures that the state transition matrices used before and after the attack come from different clusters, while intra-cluster random sampling ensures that the selected state transition matrix within the same cluster is unpredictable. Both of these factors together inject continuous non-stationarity into the attacker's observation sequence, making it difficult for frequency statistics based on historical trajectories to converge to any single state transition matrix estimate. Compared with the unstructured uniform random sampling method in existing technologies, the two-level switching protocol makes full use of the inherent grouping structure of the strategy set, providing additional defense gains.

[0154] (7) The cluster threshold sensitivity analysis revealed the non-monotonic relationship between the indicators and the defense effect: the cluster threshold with the highest patrol strategy uncertainty index is not the cluster threshold with the best defense effect. The optimal cluster threshold is between two extremes and needs to be determined by adversarial verification with the attacker's prediction error as the final criterion.

[0155] (8) A hierarchical comparative attribution system for three algorithm configurations was constructed, realizing the separable quantification of the contributions of the policy generation algorithm and the handover protocol. Experimental results show that the dual-entropy balance optimization algorithm based on clustering pattern and the two-level handover protocol configuration improve the average relative error of critical states by 26.4%–60.6% compared with the baseline configuration, and improve the importance-weighted average row error by 22.4%–53.6%. Among them, the policy generation algorithm and the two-level handover protocol each provide independent and superimposed defense gains.

Claims

1. A clustering-based method for generating and switching adversarial patrol strategies for unmanned systems, characterized in that, include: Obtain the undirected patrol topology graph of the physical environment to be patrolled, and calculate the steady-state distribution of the target based on the value parameters and attack time parameters of each node in the undirected patrol topology graph; A patrol strategy uncertainty index is defined, which is a weighted average of intra-strategy entropy and inter-strategy entropy, used to evaluate a strategy set consisting of multiple state transition matrices. Intra-strategy entropy is the arithmetic mean of the steady-state weighted entropy of each state transition matrix in the strategy set, and steady-state weighted entropy is the weighted average of the row entropy of each state transition matrix with the target steady-state distribution as the weight. Inter-strategy entropy is calculated by using the average absolute difference between the state transition matrices in the strategy set as the distance, and using agglomerative hierarchical clustering and average linkage methods to divide the strategy set into several clusters according to a predetermined clustering threshold. The Shannon entropy is calculated by using the proportion of the number of state transition matrices contained in each cluster to the total number of strategy sets as the probability distribution. Both intra-strategy entropy and inter-strategy entropy are Shannon entropy in bits. A clustering-based dual-entropy balance optimization algorithm is adopted to maximize the patrol strategy uncertainty index as the optimization objective. Under the conditions of satisfying topological constraints, row randomness constraints, element non-negativity constraints, and target steady-state distribution constraints, iterative search is performed through steady-state constraint nearest point projection, perturbation-type transition matrix generator, and simulated annealing main process to generate a strategy set containing multiple state transition matrices that satisfy the constraints. The unmanned control system executes patrol missions based on the generated policy set, and adopts the same clustering structure as when calculating the entropy between policies for a two-level switching protocol: the first level is inter-cluster switching, which arranges all clusters in random order so that the state transition matrices used before and after come from different clusters; the second level is intra-cluster sampling, which randomly and uniformly samples a state transition matrix from the current cluster, executes a random number of transition steps, and then switches to the next cluster.

2. The method according to claim 1, characterized in that, The calculation of the target steady-state distribution based on the value parameters and attack time parameters of each node in the undirected patrol topology graph includes: Obtain the value parameters and attack time parameters of each node in the undirected patrol topology graph; Based on the value parameters and attack time parameters of each node, the importance weight parameter of each node is calculated, wherein the importance weight parameter is equal to the ratio of the node's value parameter to its attack time parameter. The importance weight parameters of each node are normalized to obtain the target steady-state distribution probability value of each node. The target steady-state distribution probability value represents the expected probability of the unmanned system visiting the corresponding node during long-term patrol.

3. The method according to claim 1, characterized in that, The calculation of the entropy within the strategy includes: For each state transition matrix in the policy set, calculate the Shannon entropy of the transition probability distribution of each row as the row entropy; Using the target steady-state distribution as the weight, the row entropy is weighted and summed to obtain the steady-state weighted entropy of the state transition matrix; The internal entropy of the policy is obtained by taking the arithmetic mean of the steady-state weighted entropies of all state transition matrices in the policy set.

4. The method according to claim 1, characterized in that, The calculation of the inter-strategy entropy includes: The average of the absolute differences between the corresponding elements of the two state transition matrices is used as the policy distance. Using the policy distance as a metric, each state transition matrix in the policy set is treated as an independent cluster, and an initial distance matrix is ​​constructed. Repeat the following steps until all inter-cluster distances exceed the predetermined clustering threshold: find the two clusters with the smallest current distance; if the current distance does not exceed the predetermined clustering threshold, merge the two clusters into a new cluster, update the cluster set to remove the two merged clusters and add them to the new cluster, and use the average linking method to calculate the distance between the new cluster and the remaining clusters to update the distance matrix; the average linking method is to calculate the average of the policy distances between all state transition matrix pairs in the two clusters; When the distance between all clusters exceeds the predetermined clustering threshold, clustering is terminated, and several clusters are obtained. Count the number of state transition matrices in each cluster, and calculate the proportion of the number of state transition matrices in each cluster to the total number of policies. The Shannon entropy is calculated using the probability distribution of each cluster proportion to obtain the inter-strategy entropy: ； in, This is the clustering threshold. The number of clusters obtained from clustering. For the first The proportion of the number of state transition matrices contained in a cluster to the total number of policies.

5. The method according to claim 1, characterized in that, The weighting coefficients of the intra-strategy entropy and the inter-strategy entropy are both greater than zero; the inter-strategy entropy satisfies the following property: it takes the value of zero when all state transition matrices belong to the same cluster, and it takes the value of zero when each state transition matrix forms its own cluster. , in, The number of state transition matrices in the policy set is monotonically increasing with the increase of the predetermined clustering threshold.

6. The method according to claim 1, characterized in that, The steady-state constrained nearest point projection is used to correct the candidate matrix obtained by the perturbation-type transition matrix generator, including: The elements of the candidate matrix on the allowed edge set are extracted as candidate vectors, where the allowed edge set is the set of node pairs with edges in the undirected patrol topology graph. Under the conditions of row randomness constraint, target steady-state distribution constraint and element non-negativity constraint, construct a quadratic programming problem that minimizes the Euclidean distance between feasible vectors and candidate vectors; The optimal feasible vector is obtained by solving the quadratic programming problem using sequential quadratic programming. Mapping the optimal feasible vector back to matrix form yields a state transition matrix that satisfies the topological constraints, row stochastic constraints, and the target steady-state distribution constraints. The constraint matrix and vectors of the quadratic programming problem depend only on the target steady-state distribution and the undirected patrol topology graph, are pre-constructed once during the initialization phase and shared in subsequent projection calls.

7. The method according to claim 6, characterized in that, The perturbation-type transition matrix generator is used to generate candidate transition matrices on top of a base matrix. The inputs to the perturbation-type transition matrix generator include the base matrix, mixing coefficients, diversity factor, and entropy focusing flag. The base matrix is ​​an initial transition matrix that satisfies topological constraints and row randomness constraints. The perturbation-type transition matrix generator includes: Each row of the base matrix is ​​processed sequentially as the current row: The allowed column set of the current row is determined, which is the set of nodes in the undirected patrol topology graph that have edges with the node corresponding to the current row; normalized random noise is constructed on the allowed column set, where the element position where the node corresponding to the current row moves to itself is a self-loop position, and the self-loop position is set to zero; the current row is the current value of the corresponding row after the base matrix is ​​topologically masked, and the current row is linearly mixed with the normalized random noise according to a mixing coefficient. The linear mixing is as follows: the current row is weighted by the value after subtracting the mixing coefficient from 1, the normalized random noise is weighted by the mixing coefficient, and then the weighted current row and the weighted normalized random noise are summed. The mixing coefficient controls the degree to which the current row deviates from the base matrix. When the entropy focusing flag is enabled, the self-loop probability of the corresponding node in the current row of the base matrix is ​​weighted with a random number according to the diversity factor to determine the self-loop probability of the current row. The self-loop position is assigned the self-loop probability, and the non-self-loop elements in the current row except for the self-loop position are scaled. The scaling is to multiply the non-self-loop elements by a scaling factor, which is equal to 1 minus the self-loop probability and then divided by the sum of the non-self-loop elements, so that the sum of the elements in the current row is 1. When the entropy focus flag is disabled, the self-loop probability of the current row is set to 1 minus the sum of all non-self-loop elements in the current row; Normalize the current row to eliminate numerical errors; After processing row by row, the nearest point projection of the steady-state constraint is called to project the matrix onto the feasible region, and the state transition matrix that satisfies all constraints is output.

8. The method according to claim 7, characterized in that, The main simulated annealing process includes: Initialization phase: Setting the initial temperature T 0, generate L The n independent random basis matrices, for the nth l A random basis matrix is ​​used, and an initial state transition matrix is ​​generated by calling a perturbation-type transition matrix generator with an increasing diversity factor and alternating entropy focusing flags. The diversity factor is equal to 0.2 plus... l / L The product of 0.6, where L The number of state transition matrices in the policy set, when l When the entropy focusing flag is even, it is enabled. l When the value is odd, the entropy focusing flag is disabled; the patrol policy uncertainty index of the current policy set is calculated as a baseline value; Iterative search phase: In each round, an index is randomly selected, and the perturbation transition matrix generator is called with random parameters to regenerate the state transition matrix corresponding to the index. The remaining state transition matrices remain unchanged, forming a candidate strategy set. The patrol policy uncertainty index of the candidate policy set is calculated, and a decision on whether to accept the candidate policy set is made according to the Metropolis acceptance criterion. The Metropolis acceptance criterion is: if the patrol policy uncertainty index of the candidate policy set is greater than that of the current policy set, it is accepted unconditionally; otherwise, it is accepted with probability. Accept, among which, The difference between the patrol policy uncertainty index of the candidate policy set and the patrol policy uncertainty index of the current policy set is calculated as follows: The current temperature is set; if accepted, the current strategy set is updated to the candidate strategy set, and when the uncertainty index of the updated patrol strategy is better than the historical best value, the current strategy set is recorded as the historical best strategy set. Temperature decays according to geometric cooling schedule after each round; A set-level rebalancing is performed every set number of rounds to maintain the balance of the two target dimensions: intra-strategy entropy and inter-strategy entropy.

9. The method according to claim 1, characterized in that, This also includes determining the clustering threshold: For multiple candidate clustering thresholds, corresponding policy sets are generated respectively. The policy sets generated under each candidate clustering threshold are deployed according to the two-level switching protocol and the patrol trajectory is simulated. The attacker uses the transition frequency maximum likelihood estimator to estimate the state transition matrix of the patroller based on the observed trajectory. Using the average relative error of critical states and the importance-weighted average row error as evaluation indicators, the optimal clustering threshold is determined to maximize the attacker's prediction error. The transition frequency maximum likelihood estimator is a transition probability estimate constructed based on the frequency of occurrence of each state pair in the observed state transition sequence; the key state average relative error is the arithmetic mean of the relative errors between each row of the real state transition matrix and the corresponding rows of the estimated state transition matrix on a preset set of key nodes; the importance-weighted average row error is a weighted average of the transition probability estimation errors of each row with the target steady-state distribution as the weight.

10. The method according to claim 9, characterized in that, It also includes constructing an attribution verification system, which is used to evaluate and verify the respective defensive contributions of the policy generation algorithm and the handover protocol, including: Set a baseline configuration, use the Metropolis-Hastings algorithm to generate a policy set and uniformly and randomly sample the state transition matrix; Set up a dual-entropy balance optimization configuration, use the clustering-based dual-entropy balance optimization algorithm to generate a strategy set and uniformly and randomly extract the state transition matrix; Set up a dual-entropy balance clustering switching configuration, use the dual-entropy balance optimization algorithm based on the clustering pattern to generate a strategy set, and use the two-level switching protocol to extract the state transition matrix; Run the baseline configuration, dual-entropy balance optimization configuration, and dual-entropy balance clustering switching configuration respectively to generate the corresponding patrol trajectory; use the transition frequency maximum likelihood estimator to estimate the state transition matrix based on the patrol trajectory, and calculate the average relative error of key states and the importance-weighted average row error; The improvement includes an attack prediction error difference measurement strategy generation algorithm based on the baseline configuration to the dual-entropy balance optimized configuration, an improvement in the attack prediction error difference measurement switching protocol based on the dual-entropy balance optimized configuration to the dual-entropy balance clustering switching configuration, and a total improvement in the attack prediction error difference measurement of the baseline configuration to the dual-entropy balance clustering switching configuration. The total improvement is the sum of the improvements in the strategy generation algorithm and the switching protocol. The total improvement is used as the evaluation criterion to guide the parameter adjustment of the clustering pattern-based dual-entropy balance optimization algorithm and the deployment and verification of the two-level switching protocol.

Images