An unmanned system patrol path decision-making method in an adversarial environment

Abstract

The application discloses a kind of unmanned system patrol path decision-making method under hostile environment, belong to autonomous unmanned system and intelligent decision-making technical field.The method includes: obtaining the undirected patrol topology graph of the physical environment to be patrolled, target steady-state distribution is calculated based on the value parameter of each node, attack time parameter and degree;Define the patrol strategy unpredictability index formed by the weight of strategy internal entropy and strategy entropy PSUI , to quantify the uncertainty of strategy in micro-randomness and macro-diversity;Using simulated annealing double-entropy balance optimization algorithm, to maximize PSUI As target, generate state transition matrix strategy set under the constraints of meeting topology adjacency, line normalization and target steady-state distribution;Control unmanned system to carry out space-time decoupling patrol by random switching matrix and probabilistic transition in matrix.The application realizes the unity of patrol strategy evaluation and optimization, significantly improves the unpredictability of patrol path, and enhances the protection ability to key infrastructure.

Description

Technical Field

[0001] This invention belongs to the field of autonomous unmanned systems and intelligent decision-making technology, and specifically relates to a method for patrol path decision-making of unmanned systems in adversarial environments. Background Technology

[0002] Traditional manual patrols have inherent drawbacks such as large blind spots and weak sustainability; fixed monitoring systems, on the other hand, are easily identified and exploited by attackers with continuous observation capabilities because their paths are predictable.

[0003] Against this backdrop, autonomous unmanned systems, such as drones, unmanned vehicles, and unmanned boats, have become important tools for carrying out patrol missions on critical infrastructure due to their advantages of high mobility, strong sustainability, and low labor costs. By integrating machine learning and artificial intelligence technologies, unmanned systems can undertake most routine patrol and monitoring tasks, making up for the shortcomings of traditional protection methods, such as limited coverage and slow response speed, and achieving comprehensive and continuous protection of critical infrastructure under conditions of limited resources.

[0004] However, how to design highly unpredictable intelligent path decision-making methods for unmanned patrol systems in adversarial scenarios, making their behavior patterns difficult for attackers to predict and exploit, has become a core problem that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the technical problems raised in the background section, embodiments of the present invention provide a method for unmanned systems to make patrol paths in adversarial environments, including:

[0006] Obtain an undirected patrol topology graph of the physical environment to be patrolled. Based on the value parameters, attack time parameters, and degree of each node in the topology graph, calculate the target steady-state distribution. The target steady-state distribution represents the expected probability distribution of the patroller visiting each node during a long-term patrol.

[0007] Define the unpredictability index of patrol strategies The patroller describes the transition probabilities between nodes in the patrol topology using a state transition matrix; It consists of a weighted sum of intra-strategy entropy and inter-strategy entropy; the intra-strategy entropy is used to measure the randomness of transitions within a single state transition matrix, and the inter-strategy entropy is used to measure the diversity of differences between different state transition matrices.

[0008] A set of strategies is generated using a simulated annealing dual-entropy balance optimization algorithm, which aims to maximize the... To optimize the objective, while simultaneously satisfying the topological graph adjacency matrix constraint, the normalization constraint that the row sum of the transition probabilities is 1, and the objective steady-state distribution constraint, the solution space is iteratively searched and the state transition matrix is ​​updated according to the simulated annealing acceptance criterion, outputting a set of strategies containing multiple state transition matrices that satisfy the constraints.

[0009] The unmanned system is controlled to perform a spatiotemporally decoupled patrol mission based on different state transition matrices in the strategy set. By randomly switching different state transition matrices and randomly transitioning within a single matrix according to probability, the patrol path of the unmanned system becomes unpredictable.

[0010] This invention provides a patrol path decision-making method for unmanned systems in adversarial environments. By abstracting the physical environment to be patrolled into an undirected patrol topology graph and calculating the steady-state distribution of targets based on node value, attack time, and degree, it achieves standardized modeling of the patrol mission environment and differentiated resource allocation guidance. Furthermore, it defines a patrol strategy unpredictability index weighted by intra-strategy entropy and inter-strategy entropy. By incorporating microscopic transfer randomness and macroscopic pattern diversity into a unified quantitative framework, a clear direction is provided for the objective evaluation and optimization of strategies. The simulated annealing dual-entropy balance optimization algorithm is employed to maximize [the desired outcome] under multiple constraints. This method generates a set of state transition matrix strategies for the target, balancing the randomness within a single matrix with the differences between multiple matrices, effectively improving the unpredictability of patrol strategies. By controlling the unmanned system to randomly switch between different state transition matrices and transition within the matrix according to probability, a spatiotemporally decoupled dual random patrol mechanism is achieved, making it difficult for attackers to predict future paths from historical trajectories, significantly enhancing patrol and protection capabilities in adversarial environments. At the same time, this method focuses on strategy modeling and analysis, is platform-independent, and can be applied to various autonomous unmanned patrol systems such as UAVs, unmanned vehicles, and unmanned boats, demonstrating good versatility and application prospects. Attached Figure Description

[0011] Figure 1 A flowchart of a patrol path decision method for an unmanned system in an adversarial environment provided by an embodiment of the present invention;

[0012] Figure 2 A schematic diagram illustrating a Markov chain containing five nodes;

[0013] Figure 3 This is a visualization diagram of a portion of the state transition matrix generated by the simulated annealing dual-entropy balance optimization algorithm in an embodiment of the present invention;

[0014] Figure 4 This is a comparison curve of the simulated annealing dual-entropy balance optimization algorithm and other algorithms on the patrol strategy unpredictability index in this embodiment of the invention;

[0015] Figure 5 This is a flowchart of the simulated annealing dual-entropy balance optimization algorithm in an embodiment of the present invention;

[0016] Figure 6 This is a comparison curve of the average relative error of the key states predicted by the attacker in an embodiment of the present invention;

[0017] Figure 7 This is a comparison curve of the maximum row error weighted by the importance predicted by the attacker in an embodiment of the present invention. Detailed Implementation

[0018] Traditional manual patrols suffer from inherent drawbacks such as large blind spots and weak sustainability, while fixed monitoring systems, due to their predictable path patterns, are easily identified and exploited by attackers with continuous observation capabilities. Against this backdrop, autonomous unmanned systems, such as drones, unmanned vehicles, and unmanned boats, have become important tools for patrolling critical infrastructure due to their high mobility, strong sustainability, and low manpower costs. However, designing highly unpredictable intelligent path decision-making methods for unmanned patrol systems in adversarial scenarios, making their behavior patterns difficult for attackers to predict and exploit, has become a core problem that urgently needs to be solved in this field.

[0019] Against this backdrop, utilizing unmanned systems for intelligent patrol missions offers a new strategic opportunity to address the aforementioned challenges. By integrating machine learning and artificial intelligence technologies, unmanned systems can undertake most routine patrol and monitoring tasks, not only compensating for the limited coverage and slow response times of traditional protection methods, but also achieving comprehensive and continuous protection of critical infrastructure under resource constraints. Especially in adversarial scenarios, the introduction of randomized decision-making mechanisms makes patrol paths highly unpredictable, significantly improving the overall effectiveness of protecting critical infrastructure.

[0020] In unmanned system patrol missions under adversarial conditions, the unpredictability of patrol paths is a core element in measuring the effectiveness of protection. However, existing research still has room for improvement in the following aspects: (1) A unified quantitative index and optimization target for unpredictability have not yet been formed. The single-step randomness within a strategy and the diversity of cross-matrix strategies are usually handled separately and not incorporated into the same optimization framework for collaborative design. This not only makes it difficult to objectively compare different patrol strategies, but also makes the algorithm optimization lack a clear quantitative direction; (2) The attacker's prediction behavior is usually not specifically modeled, and there is a lack of calculation methods for the attacker to estimate the state transition matrix of the patroller based on historical observation sequences, which makes it impossible to effectively verify the actual protection effect of the patrol strategy from an adversarial perspective; (3) The node importance information is mainly used to generate the steady-state distribution of the target and has not yet been incorporated into the measurement system of the attacker's prediction error, making it difficult for the evaluation results to highlight the protection priority of key nodes with high value and short attack time.

[0021] To address the aforementioned areas for improvement, this invention proposes a systematic solution from four dimensions: evaluation index design, optimization algorithm development, attacker behavior modeling, and comprehensive evaluation framework construction. It also introduces for the first time a patrol strategy unpredictability index (…). This paper proposes a method to unify intra-strategy entropy (single-matrix randomness) and inter-strategy entropy (cross-matrix diversity) into a single index through a weighted linear combination. This index serves as both an objective evaluation standard and an optimization objective, achieving a unity of evaluation and optimization. A simulated annealing dual-entropy balance optimization algorithm (SADEBO) was developed to maximize the desired outcome while satisfying topological constraints, row normalization constraints, and the objective steady-state distribution constraints. Two types of entropy metrics are optimized simultaneously for the target. A maximum likelihood estimation attacker model based on counting is constructed, which estimates the transition matrix solely by observing historical state sequences without relying on any prior information, realistically simulating the behavior of a rational attacker. A multi-dimensional error evaluation system based on key node screening and node importance weighting is designed, directly incorporating node importance into error calculation, focusing on evaluating the protection effectiveness of high-value key nodes. Theoretically, it has been proven... The positive correlation with the attacker's prediction error means that, in practical applications, by using... To optimize the design of the target guidance algorithm and improve the overall protection effect of unmanned systems when performing patrol missions.

[0022] The following will combine Figure 1 This invention describes a method for determining patrol paths for unmanned systems in adversarial environments, as provided in an embodiment of the present invention. In this embodiment, "patrolman" refers to an autonomous unmanned system performing patrol missions, including but not limited to drones, unmanned vehicles, and unmanned surface vessels. Because this invention focuses on modeling and analyzing the patrol strategy itself, rather than targeting a specific hardware platform, the established model is platform-independent and applicable to various autonomous unmanned patrol systems.

[0023] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for unmanned system patrol path decision-making in an adversarial environment, which may include the following steps:

[0024] S110: Obtain the undirected patrol topology graph of the physical environment to be patrolled. Based on the value parameters, attack time parameters, and degree of each node in the topology graph, calculate the steady-state distribution of the target.

[0025] The target steady-state distribution represents the expected probability distribution of a patroller visiting each node during a long-term patrol.

[0026] In this embodiment, the physical environment to be patrolled is first discretized and modeled. The various locations that the unmanned system needs to patrol and monitor in the real environment are abstracted as nodes in a topology graph, and the reachable paths between locations are abstracted as edges in the topology graph, thereby constructing an undirected patrol topology graph. Each node in the topology graph represents a specific patrol location. The set representing vertices (patrol positions), each node in the graph This represents locations in a real-world environment where unmanned systems need to be patrolled and monitored. n This indicates the total number of patrol positions. The set representing the edges, Represents the node Transfer to node The probability satisfies ;like Figure 2 This represents a Markov chain containing five nodes.

[0027] For each node in the topology graph, obtain its value parameter and attack time parameter. The value parameter reflects the importance of the node in the overall protection task, and the attack time parameter reflects the length of time required for an attacker to successfully attack the node. Simultaneously, obtain the degree of each node in the topology graph, i.e., the number of neighboring nodes directly connected to that node.

[0028] Based on the value parameters, attack time parameters, and degree of each node, the steady-state distribution of the target is calculated. The steady-state distribution of the target represents the expected probability distribution of the patroller's visits to each node during long-term patrols, and is used to guide the generation of subsequent patrol strategies, so that high-value nodes with short attack times receive a higher probability of being visited.

[0029] To ensure clarity, the calculation process for the target steady-state distribution will be explained in detail in the following embodiments.

[0030] S120 defines the patrol strategy unpredictability index. .

[0031] Specifically, the movement of the patroller between nodes in the topology is described by a state transition matrix. The elements in the state transition matrix represent the probability of the patroller moving from one node to another, and the sum of the probabilities of moving from any node to all possible next nodes is 1.

[0032] To quantify the unpredictability of patrol strategies, this embodiment defines a patrol strategy unpredictability index. . It is a quantitative bridge between theoretical and practical patrol effectiveness: Transforming the abstract concept of "unpredictability" into measurable numerical values; It consists of a weighted sum of intra-strategy entropy and inter-strategy entropy. Intra-strategy entropy measures the randomness of transitions within a single state transition matrix, reflecting the degree of uncertainty when a patroller performs a single-step transition under a single matrix, thus improving the unpredictability of patrol paths in the short term. Inter-strategy entropy measures the diversity of differences between different state transition matrices, reflecting the degree of change in the overall patrol pattern when the patroller switches between different matrices, ensuring the change of patrol pattern over long-term scales. It comprehensively reflects the uncertainty of the patrol strategy set at both the micro-level of randomness and the macro-level of diversity. The higher the score, the stronger the unpredictability of the patrol strategy.

[0033] To ensure clarity, the following embodiments will explain the calculation steps for intra-strategy entropy and inter-strategy entropy, as well as the patrol strategy unpredictability index. The calculation formula will be explained in detail.

[0034] S130, uses the simulated annealing dual-entropy balance optimization algorithm to generate a strategy set. The simulated annealing dual-entropy balance optimization algorithm aims to maximize... To optimize the objective, while simultaneously satisfying the topological graph adjacency matrix constraint, the normalization constraint that the row sum of the transition probabilities is 1, and the objective steady-state distribution constraint, the solution space is iteratively searched and the state transition matrix is ​​updated according to the simulated annealing acceptance criterion, outputting a set of policies containing multiple state transition matrices that satisfy the constraints.

[0035] Specifically, in the definition As the optimization objective, this embodiment employs a simulated annealing dual-entropy balance optimization algorithm to generate the patrol strategy set. This algorithm aims to maximize... To optimize the objective, the following constraints must be satisfied during the search process: topological graph adjacency matrix constraint, i.e., state transitions can only occur between nodes with edges in the topological graph; normalization constraint that the sum of row sums of transition probabilities is 1, i.e., the sum of each row element in each state transition matrix is ​​equal to 1; and target steady-state distribution constraint, i.e., the actual steady-state distribution corresponding to the generated policy set should be consistent with the target steady-state distribution calculated in step S110.

[0036] During the algorithm execution, the solution space is iteratively searched and the state transition matrix is ​​continuously updated according to the simulated annealing acceptance criterion. Finally, the output is a policy set containing multiple state transition matrices that satisfy the above constraints.

[0037] S140, the unmanned system is controlled to perform a spatiotemporally decoupled patrol mission according to different state transition matrices in the strategy set. By randomly switching different state transition matrices and randomly transitioning within a single matrix according to probability, the patrol path of the unmanned system becomes unpredictable.

[0038] After obtaining the set of strategies, they are deployed to the autonomous unmanned system performing patrol missions. When performing patrol missions, the unmanned system randomly switches between different state transition matrices to adopt different patrol modes at different times; at the same time, within each state transition matrix, random transitions are performed between nodes according to the probability distribution defined by the matrix.

[0039] By randomly switching between different matrices and randomly transferring within a single matrix according to probability, the patrol path of the unmanned system exhibits a high degree of unpredictability on both short-term and long-term scales. This effectively increases the difficulty for attackers to predict future patrol paths by observing historical trajectories, thereby improving the overall protection effect on critical infrastructure.

[0040] This invention provides a patrol path decision-making method for unmanned systems in adversarial environments. By abstracting the physical environment to be patrolled into an undirected patrol topology graph and calculating the steady-state distribution of targets based on node value, attack time, and degree, it achieves standardized modeling of the patrol mission environment and differentiated resource allocation guidance. Furthermore, it defines a patrol strategy unpredictability index weighted by intra-strategy entropy and inter-strategy entropy. By incorporating microscopic transfer randomness and macroscopic pattern diversity into a unified quantitative framework, a clear direction is provided for the objective evaluation and optimization of strategies. The simulated annealing dual-entropy balance optimization algorithm is employed to maximize [the desired outcome] under multiple constraints. This method generates a set of state transition matrix strategies for the target, balancing the randomness within a single matrix with the differences between multiple matrices, effectively improving the unpredictability of patrol strategies. By controlling the unmanned system to randomly switch between different state transition matrices and transition within the matrix according to probability, a spatiotemporally decoupled dual random patrol mechanism is achieved, making it difficult for attackers to predict future paths from historical trajectories, significantly enhancing patrol and protection capabilities in adversarial environments. At the same time, this method focuses on strategy modeling and analysis, is platform-independent, and can be applied to various autonomous unmanned patrol systems such as UAVs, unmanned vehicles, and unmanned boats, demonstrating good versatility and application prospects.

[0041] exist Figure 1 Based on the illustrated embodiment, the following will discuss... Figure 1 The calculation process of the stable distribution of the target is explained in detail.

[0042] First, obtain the nodes in the patrol topology map. Value parameters and attack time parameters Among them, value parameters Representation Nodes The importance of a facility in the overall defense mission can be pre-set based on factors such as its strategic significance and asset value; attack timing parameters. Indicates that the attacker successfully attacked the node. The required time reflects the vulnerability of the node. These parameters can be configured according to the specific needs of the actual application scenario.

[0043] Secondly, calculate each node Weight parameters The calculation formula is:

[0044]

[0045] This weight parameter This comprehensively reflects the relationship between a node's value and the ease of attack: the higher the value, the shorter the attack time. The larger the value, the higher the priority that the node should receive in the allocation of patrol resources.

[0046] Next, obtain each node. Degree in the topological graph The degree mentioned For nodes The number of directly connected neighbor nodes reflects the connectivity and reachability of a node in the topology graph. In the Markov chain model, the same... Nodes with higher degrees are more likely to maintain a higher steady-state access probability. Therefore, incorporating degree into the target steady-state distribution can make the calculation results more consistent with the actual constraints of the graph topology.

[0047] Finally, compute nodes The target steady-state distribution probability value is calculated using the following formula:

[0048]

[0049] in, The denominator represents the total number of nodes in the topology graph, and the denominator represents the sum of all nodes. From 1 to of Summation is performed to normalize the molecules, so that the sum of the target steady-state distribution probabilities of all nodes is 1.

[0050] The target steady-state distribution is calculated through the above steps. This represents the number of nodes visited by the patroller during a long patrol. The expected probability distribution is used to determine the resource allocation for generating subsequent patrol strategies, ensuring that high-value nodes with short attack times receive higher patrol access frequencies, thereby optimizing the overall protection effect.

[0051] exist Figure 1 Based on the illustrated embodiment, this embodiment further... Figure 1 The steps for calculating the entropy within the strategy in the illustrated embodiment are explained in detail.

[0052] Policy entropy measures the randomness of transitions within a single state transition matrix in a policy set. Its calculation includes the following steps:

[0053] First, for each state transition matrix in the policy set Q Calculate the first in the matrix row entropy . No. The row entropy represents the number of rows a patroller moves from a node. The uncertainty of the probability distribution of the departure and transfer to other nodes is calculated by the following formula:

[0054]

[0055] in, State transition matrix From the node Transfer to node The probability, The total number of nodes in the topological graph. Row entropy. The larger the value, the more likely it is to originate from the node. The more dispersed the starting transfer directions are, the more difficult they are to predict; conversely, if the transfer probability is concentrated on a few nodes, the row entropy value is smaller, and the transfer behavior is more deterministic.

[0056] Secondly, the target steady-state distribution calculated in step S110 is used. As weights, for the state transition matrix The row entropy of each row in the matrix is ​​weighted and summed to obtain the matrix. Weighted average row entropy The calculation formula is:

[0057]

[0058] Among them, the target steady-state distribution This reflects the patroller's expected frequency of visits to each node during long-term patrols. Weighted averaging of the weights ensures that nodes accessed more frequently during actual patrols contribute more to the overall uncertainty measure, thus more accurately reflecting the degree of randomness in the actual execution of the patrol strategy. Weighted average row entropy. Integrating matrix The uncertainty information of all rows is used to characterize the overall uncertainty of a single state transition matrix.

[0059] Finally, the weighted average row entropy of all state transition matrices in the policy set Q is calculated. Calculate the arithmetic mean to obtain the policy internal entropy. The calculation formula is:

[0060]

[0061] in, The size of the policy set, representing the number of state transition matrices in the policy set. Policy internal entropy. It reflects the average level of randomness of all state transition matrices in the policy set. The higher the value, the more difficult it is for an attacker to predict the patroller's single-step transition behavior within a single state transition matrix.

[0062] Through the above three-level progressive calculation (row entropy → weighted average row entropy → set arithmetic mean), the policy internal entropy quantifies the micro-randomness within a single matrix into a single numerical index, serving as an unpredictability index for subsequent patrol strategies. This provides a foundation for computation and optimization of patrol strategies.

[0063] This embodiment achieves precise quantification of the randomness of transitions within a single state transition matrix through the aforementioned method for calculating the internal entropy of the strategy. This method has the following advantages: First, using information entropy as a measure of row entropy objectively reflects the uniformity of the transition probability distribution originating from each node, with sufficient theoretical basis. Second, introducing the target steady-state distribution as a weighting factor ensures that the contribution of different nodes to the overall randomness measurement is consistent with the actual patrol resource allocation strategy, making the evaluation results more aligned with the actual patrol mission requirements. Third, by calculating the arithmetic mean of all matrices in the strategy set, the potential random bias of a single matrix is ​​eliminated, enabling the internal entropy of the strategy to stably characterize the overall microscopic unpredictability level of the strategy set, providing a reliable quantitative basis for the horizontal comparison and optimization of patrol strategies.

[0064] exist Figure 1 Based on the illustrated embodiment, this embodiment further... Figure 1 The steps for calculating the inter-strategy entropy in the illustrated embodiment are explained in detail.

[0065] Inter-policy entropy measures the diversity of differences between state transition matrices in a policy set, that is, the degree of change in the overall patrol pattern when a patroller switches state transition matrices at different time stages. Its calculation process includes the following steps:

[0066] First, for any two distinct state transition matrices in the policy set Q. and Calculate the mean absolute difference between the two. .in, For the policy set Q, the first... A state transition matrix, For the policy set Q, the first... A state transition matrix; Indicates the first From node in each state transition matrix Transfer to node The probability, Indicates the first From node in each state transition matrix Transfer to node The probability of the mean absolute difference. The calculation formula is:

[0067]

[0068] in, Let be the total number of nodes in the topology graph. This formula calculates the average of the absolute differences between the two matrices at all corresponding element positions, thus measuring the degree of difference in the probability distributions between the two state transition matrices as a whole. The larger the value, the more significant the difference in the transition probability distribution patterns between the two matrices, and the greater the difference in path characteristics exhibited by the patroller when performing patrol tasks based on these two matrices.

[0069] Secondly, compute the differences between all distinct matrix pairs in the strategy set Q. The arithmetic mean of the strategies is used to obtain the inter-strategy entropy. The policy set Q contains a total of There are n state transition matrices, and the total number of different matrix pairs is . For all unique matrix pairs (i.e. )of The summation of the values, divided by the total number of matrix pairs, is calculated using the following formula:

[0070]

[0071] Among them, coefficient This is the reciprocal of the total number of matrix pairs, used to calculate the arithmetic mean. Inter-strategy entropy. It comprehensively reflects the average difference level in transition probability patterns among all state transition matrices in the strategy set. The higher the value, the more diverse the transition probability distribution of different matrices in the strategy set generated by the simulated annealing dual-entropy balance optimization algorithm, and the greater the difference in patrol modes adopted by the patroller in different time periods.

[0072] Through the above calculations, inter-policy entropy quantifies the macroscopic diversity of the policy set into a single numerical indicator. In actual adversarial scenarios, attackers identify and predict patrol patterns through long-term observation. Higher inter-policy entropy means a richer variety of patterns exhibited by patrollers over time, making it more difficult for attackers to deduce stable patterns from historical observations, thus effectively increasing the difficulty of prediction. Inter-policy entropy and intra-policy entropy together constitute the patrol policy unpredictability index. The two core dimensions provide a quantitative basis for the comprehensive evaluation and optimization of patrol strategies.

[0073] This embodiment achieves precise quantification of the diversity of differences between different state transition matrices in a policy set through the aforementioned method for calculating inter-policy entropy. This method has the following advantages: First, using the mean absolute difference as the distance metric between matrices is intuitive and sensitive to the overall shift in the probability distribution, effectively capturing the differences in global transition behavior between two patrol policies; Second, by analyzing all matrix pairs... The arithmetic mean of the values ​​eliminates the potential bias of comparing a single matrix, allowing the inter-strategy entropy to stably reflect the overall macroscopic diversity level of the strategy set. Third, the introduction of inter-strategy entropy extends the assessment of the unpredictability of patrol strategies from the randomness within a single matrix to the pattern differences between multiple matrices. This compensates for the shortcomings of existing methods that only focus on microscopic randomness while ignoring macroscopic diversity, providing key technical support for generating truly unpredictable spatiotemporal decoupled patrol strategies.

[0074] exist Figure 1 Based on the illustrated embodiment, this embodiment further... Figure 1 The patrol strategy unpredictability index in the illustrated embodiment The definition and calculation method are explained in detail.

[0075] In unmanned patrol missions under adversarial conditions, it is necessary to quantitatively assess the unpredictability of patrol strategies in order to objectively compare different strategies and provide clear directions for strategy optimization. To this end, this invention defines a patrol strategy unpredictability index. The calculation formula is as follows:

[0076]

[0077] in, The policy entropy measures the randomness of transitions within a single state transition matrix. is the inter-strategy entropy, used to measure the diversity of differences between different state transition matrices; The weighting coefficients are the internal entropy of the strategy. These are the weighting coefficients for the inter-strategy entropy. By adjusting these two weighting coefficients, the relative importance of micro-randomness and macro-diversity in unpredictability assessment can be flexibly configured according to actual application needs.

[0078] It is a comprehensive evaluation index of the quality of a set of strategies, reflecting the uncertainty of the strategy set in two dimensions: transition randomness and diversity of differences. Specifically, the higher the intra-strategy entropy, the more difficult it is to predict the single-step transition of the patroller within a single state transition matrix; the higher the inter-strategy entropy, the greater the difference between the state transition matrices switched by the patroller at different time periods, and the more difficult it is for attackers to detect long-term patrol patterns. The uncertainty of the two dimensions mentioned above is integrated into a single value by weighted summation. The higher the score, the stronger the overall unpredictability of the strategy set at both the micro and macro levels.

[0079] In practical applications, It can serve as an evaluation standard for objectively comparing patrol strategies generated by different algorithms; it can also serve as an optimization objective to guide the design of patrol strategy generation algorithms, enabling the algorithms to maximize the unpredictability of the strategy set while satisfying topological constraints and steady-state distribution constraints, thereby enhancing the overall protection effect on critical infrastructure.

[0080] Furthermore, to verify the advantages of the patrol strategy generated by this invention in terms of unpredictability, the simulated annealing dual-entropy balance optimization algorithm (SADEBO) proposed in this invention is compared with the multivariate stochastic adaptive Markov steady-state matrix algorithm (DRAMSM) and the multi-stage entropy-driven stochastic matrix optimization algorithm (MEDRO). The experiment uses the same patrol topology and target steady-state distribution, and the above three algorithms are used to generate strategy sets containing different numbers of state transition matrices, and the corresponding patrol strategy unpredictability indices are calculated. .

[0081] The generated visualization matrix is ​​as follows Figure 3 As shown, the experimental results are as follows: Figure 4 As shown, the horizontal axis represents the number of state transition matrices in the policy set (i.e., Matrix Count), and the vertical axis is... The red curve represents the result of the SADEBO algorithm of this invention, the green curve represents the result of the DRAMSM algorithm, and the blue curve represents the result of the MEDRO algorithm. As can be seen from the graph, with the same number of matrices, the algorithm of this invention... The values ​​are all higher than the other two algorithms, indicating that the patrol strategy generated by this invention has better unpredictability in both micro-randomness and macro-diversity, and can effectively improve the patrol and protection capabilities in adversarial scenarios.

[0082] exist Figure 1 Based on the illustrated embodiment, as one implementation of this invention, generating a strategy set using the simulated annealing dual-entropy balance optimization algorithm may include the following steps:

[0083] The first step is to input the topological adjacency matrix of the Markov chain, the target steady-state distribution, the random seed, and the number of target matrices to be generated.

[0084] In this step, the Markov chain topological adjacency matrix derived from the undirected patrol topology graph constructed in step S110 is first obtained. This adjacency matrix is ​​a... A binary matrix, where Let be the total number of nodes in the topological graph, and let be the th node in the matrix. Line 1 When an element in a column has a value of 1, it represents a node. With nodes There are walkable edges between the nodes, and a value of 0 indicates that there is no direct path between the two nodes. This adjacency matrix provides a topological constraint for the generation of the subsequent state transition matrix, that is, non-zero transition probabilities are only allowed at positions in the state transition matrix where the corresponding adjacency matrix element is 1.

[0085] Simultaneously, the target steady-state distribution calculated in step S110 is input. The target steady-state distribution is a distribution of length [missing information]. A probability vector such that the sum of all its elements is 1, and its i-th element is... element This indicates that the patroller visited the node during a long patrol. The expected probability. The target steady-state distribution provides a distribution constraint benchmark for the algorithm's optimization process.

[0086] In addition, the input includes a random seed to control the randomness of the algorithm, ensuring the reproducibility of the algorithm's results; and the number of target matrices to be generated, which determines the number of state transition matrices in the final output policy set.

[0087] The second step is the initialization phase, which generates the target matrix number and the initial state transition matrix. The generation process includes: randomly generating self-loop probabilities for each node, constructing off-diagonal elements using detailed balance conditions, and normalizing all rows to ensure that the generated matrix is ​​a Markov transition matrix.

[0088] During the initialization phase, the algorithm generates a number of initial state transition matrices that satisfy the basic constraints, each matrix having a dimension of 1. The specific generation process for each initial matrix to be generated is as follows:

[0089] First, for each node in the topology graph A self-loop probability is randomly generated. The self-loop probability represents the probability that the patroller has a loop from the node. Return directly to the node after departure The probability of can be obtained by uniformly random sampling within the interval (0,1).

[0090] Secondly, the off-diagonal elements in the matrix are constructed using the detailed balance condition. The detailed balance condition is an important property in Markov chain theory, stated as follows: for any two nodes... and If there exists from arrive The transfer should satisfy the following conditions: During the initialization phase, for node pairs marked as having an edge in the adjacency matrix... This condition allows initial values ​​to be assigned to the transition probabilities in both directions, so that the generated matrix has good symmetry and theoretical convergence in the initial state.

[0091] Finally, row normalization is performed on each row of the matrix. Specifically, for the first row of the matrix... Calculate the sum of all elements in the row. Then divide each element in that row by This process ensures that the sum of all elements in the row equals 1. The matrix processed by the above steps satisfies the definition of a Markov transition matrix and can be included in the policy set as the initial state transition matrix.

[0092] The third step: Simulated annealing optimization phase. After initializing the annealing simulation parameters, the following iterative process is executed: Randomly select matrix elements in the current state transition matrix for adjustment, maintaining a normalization constraint where the row sum is 1 after adjustment. Calculate the actual steady-state distribution corresponding to the adjusted matrix, and calculate the error between it and the target steady-state distribution. This aims to maximize... To optimize the objective, the algorithm decides whether to accept the current adjustment based on the simulated annealing acceptance criterion. This process is repeated iteratively until the algorithm converges to the target steady-state distribution or reaches the preset maximum number of iterations.

[0093] Specifically, after generating the initial matrix, the simulation annealing optimization phase begins. First, the relevant parameters required for the annealing simulation are initialized, including the initial temperature. Termination temperature Annealing rate coefficient and the number of iterations at each temperature. A higher initial temperature helps the algorithm to fully explore the solution space in the early stages, while the search behavior of the algorithm tends to stabilize and converge as the temperature gradually decreases.

[0094] In each iteration at each temperature, perform the following operations:

[0095] First, randomly select an element from a state transition matrix in the current policy set. Adjustments can be made. These adjustments can be achieved by randomly increasing or decreasing the value of the element by a certain margin, or by generating new candidate values ​​using random perturbation. It should be noted that the selected matrix elements... It must correspond to a position in the adjacency matrix where the value is 1, i.e., a node. With nodes If an edge exists between the points, then the probability value at that position is always zero and does not participate in the adjustment. After adjustment, the probability value of the matrix at that position is... The row is re-normalized to ensure that the sum of all transition probabilities in the row is still 1, thus maintaining the normalization constraint that the row sum is 1.

[0096] Secondly, the actual steady-state distribution corresponding to each state transition matrix in the adjusted policy set is calculated. For each state transition matrix, its steady-state distribution can be approximated by solving its eigenvectors or using a power iteration method. After obtaining the steady-state distribution of each matrix, its relationship with the input target steady-state distribution is calculated. The error between them. The error can be measured by mean squared error, KL divergence, or other suitable distribution distance measures, which are used to quantify the degree of deviation of the current strategy set from the target resource allocation strategy.

[0097] Secondly, to maximize the unpredictability index of patrol strategies. To optimize the objective, the acceptance of this matrix element adjustment is determined based on the simulated annealing acceptance criterion. Specifically, the strategy set before and after the adjustment is calculated. Change .like That is, adjustment makes PSUI If there is any improvement, then this adjustment will be accepted unconditionally; if... That is, adjustment leads to If it decreases, then by probability... Accept this adjustment, among which This represents the current annealing temperature. This acceptance criterion allows the algorithm to accept poor candidate solutions with a certain probability during the search process, thereby effectively avoiding getting trapped in local optima and enhancing the algorithm's global optimization ability.

[0098] Complete the current temperature After each iteration, the temperature is updated according to the annealing rate coefficient. This iterative process is repeated until the current temperature drops to the termination temperature. The following occurs either when the error between the actual steady-state distribution and the target steady-state distribution converges to within a preset threshold, or when the preset maximum number of iterations is reached.

[0099] The fourth step: Output the set of state transition matrices that satisfy the adjacency matrix constraint, the row sum of 1 constraint, and the target steady-state distribution constraint.

[0100] Once the algorithm meets the termination condition, it outputs the final set of state transition matrices. Each matrix in this set satisfies the following constraints: First, an adjacency matrix constraint, meaning that non-zero transition probabilities in the matrix only occur between node pairs with edges in the topological graph; second, a normalization constraint where the sum of each row element in each matrix is ​​1, ensuring it is a valid Markov transition matrix; third, a target steady-state distribution constraint, meaning that the actual steady-state distribution corresponding to each matrix in the policy set is consistent with the preset target steady-state distribution or the error is within an allowable range. Furthermore, under the premise of satisfying the above constraints, this policy set has the maximum value obtained through simulated annealing optimization. PSUI The value takes into account both the randomness of transitions within a single state transition matrix and the diversity of differences between different matrices, providing a high-quality decision-making basis for unmanned systems to perform unpredictable patrol missions.

[0101] To more intuitively demonstrate the execution flow of the simulated annealing dual-entropy balance optimization algorithm, this embodiment of the invention also provides a flowchart of the algorithm, such as... Figure 5 As shown. Figure 5 It clearly demonstrates the complete process from input, initialization, iterative optimization to final output, making it easy for those skilled in the art to understand and implement.

[0102] This embodiment, through the specific implementation process of the simulated annealing dual-entropy balance optimization algorithm described above, has the following beneficial effects: First, it formalizes the patrol strategy generation problem into an optimization problem under multiple constraints. By clarifying the complete process of input, initialization, iterative optimization, and output, the strategy generation process becomes standardized and reproducible. Second, in the initialization stage, detailed balance conditions are used to construct the initial matrix, giving the initial solution good theoretical properties, effectively reducing the search overhead of subsequent optimization processes, and improving the algorithm's convergence efficiency. Third, in the simulated annealing optimization stage, by simultaneously considering the error between the actual steady-state distribution and the target steady-state distribution, as well as... The optimization objective achieves synergistic optimization of resource allocation rationality and path unpredictability, avoiding strategy deviations that may be caused by single-objective optimization; fourth, the output strategy set can be directly deployed in various unmanned patrol systems without additional online computing overhead, facilitating rapid application and integration in practical engineering.

[0103] exist Figure 1 Based on the illustrated embodiment, as a specific implementation of this invention, the method further includes an evaluation step of the attacker's predicted behavior. By simulating the attacker's observation and estimation process of the patroller's behavioral patterns, the actual protective effect of the generated patrol strategy in adversarial scenarios can be quantitatively evaluated.

[0104] In this embodiment, the attacker employs a counting-based estimator algorithm to estimate the patroller's state transition matrix. This algorithm is entirely driven by observational data, requiring no prior information assumptions and relying solely on continuous observation of the patroller's state sequence to learn the state transition patterns. The specific implementation includes the following steps:

[0105] Step a1, the input to the counting-based estimator algorithm is the observed state sequence. and number of states The output is the estimated state transition matrix. .

[0106] The input to the counting-based estimator algorithm is a sequence of patroller states obtained by the attacker through long-term observation. ,in , indicating at time The node number where the observed patrolman is located, The length of the observation sequence, Let be the total number of nodes in the patrol topology. The algorithm's output is the state transition matrix estimated by the attacker. The matrix is ​​a A square matrix, in which elements This indicates the attacker's estimate of the patrollers from the node. Transfer to node The probability of.

[0107] Step a2: Statistically analyze the transitions at each step in the state sequence. Number of times, Indicates from state to state The number of transfers.

[0108] Specifically, the attacker targets the observed state sequence Perform a traversal and count the transitions at each step in the sequence. The number of times it appears. For any node pair ,definition For the patrollers in the observation sequence from the node Transfer to node The total number of times. If there are two consecutive states at different times in the sequence, respectively and Then the corresponding The count is incremented by 1. This statistical process relies solely on the observed sequence of states and does not involve any assumptions about the patroller's internal strategy.

[0109] Step a3, for each state Calculate the total number of transitions starting from this state. ,in, For the first The sum of lines.

[0110] For each state Calculate the total number of transitions starting from this state. The total number of times is the [number]th ... The sum of all elements in the row, i.e.:

[0111]

[0112] in, This reflects the patroller's movement from node in the observation sequence. The total frequency of transitions. This value will be used as the denominator in subsequent transition probability estimates to normalize each row.

[0113] Step a4, for each state pair The estimated transition probability is .

[0114] This formula is a maximum likelihood estimation based on counting, which means: the observed slave nodes To the node The number of transfers from node The proportion of the total number of transitions initiated is used as an estimate of the true transition probability. As the observation sequence becomes sufficiently long, this estimate will gradually approach the actual state transition probability used by the patroller.

[0115] Step a5: If a certain state has never appeared as a starting state in the observation sequence, then the transition probability of that state is set to a uniform distribution.

[0116] In actual observation, a certain state may exist. In cases where a state never appears as the starting state in the observation sequence, meaning that the state is never visited by the patroller, or only appears as the last state in the sequence without any subsequent transitions, then... Therefore, probability estimation cannot be performed using the formula in step a4. In such cases, the attacker sets the transition probability corresponding to this state to a uniform distribution, i.e., for all... ,make This approach reflects the attacker's uncertainty regarding the state transition behavior in the absence of observational information.

[0117] Through steps a1 to a5 above, the attacker can construct an estimate of the patroller's state transition matrix based on observations of the patroller's state sequence. This estimation matrix provides a foundation for subsequent calculations of attacker prediction errors and evaluation of the actual protective effectiveness of patrol strategies.

[0118] This embodiment achieves quantitative verification of the protective effect of patrol strategies from an adversarial perspective through the aforementioned evaluation steps for attacker prediction behavior. This evaluation method has the following advantages: First, the attacker model is entirely driven by observational data and does not rely on any prior knowledge of the patrol strategy, enabling it to realistically simulate the behavior patterns of rational attackers in actual adversarial scenarios, resulting in highly reliable and valuable evaluation results. Second, by statistically analyzing the number of transitions and employing the maximum likelihood estimation method, attackers can effectively estimate the state transition matrix with low computational complexity, facilitating rapid deployment and iterative verification in simulations and actual tests. Third, filling in initial states not appearing in the observation sequence with a uniform distribution conforms to the maximum entropy principle in information theory, avoiding estimation bias caused by data sparsity and ensuring the integrity and usability of the estimated matrix. Fourth, this evaluation step can be integrated with the aforementioned patrol strategy generation process, forming a complete closed loop of "strategy generation—adversarial evaluation—effect verification," providing an objective and quantitative evaluation method for iterative optimization and horizontal comparison of patrol strategies.

[0119] Based on the above embodiments, as a specific implementation of the present invention, the evaluation step further includes calculating the error of the state transition matrix predicted by the attacker:

[0120] Step b1, determine the patroller's true state transition matrix as follows: The attacker estimates the state transition matrix as follows: .

[0121] Specifically, the patroller executes its patrol mission according to the strategy set generated by the simulated annealing dual-entropy balance optimization algorithm, and the actual state transition matrix used internally is denoted as... The attacker uses the aforementioned counting-based estimator algorithm to calculate the estimated state transition matrix based on the observed state sequence, denoted as... Both matrices are The square array, in which This represents the total number of nodes in the patrol topology graph.

[0122] Step b2: Determine the weight parameters of each node based on its value parameters and attack time parameters, and sort all nodes in descending order according to their weight parameters, selecting the top... These nodes constitute the key node set. .

[0123] Specifically, the weight parameters of each node are determined based on the value parameters and attack time parameters of each node in the patrol topology diagram. Weight parameters Comprehensive reflection of nodes Importance and vulnerability in the protection task: The higher the value and the shorter the time required for an attack, the larger the weight parameter, indicating that the node should receive more attention in error assessment.

[0124] After obtaining the weight parameters of all nodes, allocate all nodes according to the weight parameters. Sort in descending order and select the first few items in the list. These nodes constitute the key node set. .in, The preset number of key nodes can be configured according to the needs of the actual application scenario. Key Node Set This set concentrates the most important and vulnerable nodes in the protection mission, and subsequent error calculations will focus on this set to highlight the assessment of the protection effectiveness against high-value targets.

[0125] Step b3, calculate the true state transition matrix The Rows and estimation of state transition matrix The Differences between rows, and select a preset norm type. Measure the difference.

[0126] For the set of key nodes Each node in Obtain the patroller's true state transition matrix The row vector And the attacker estimates the state transition matrix. The row vector This row vector represents the vector from the node. The probability distribution of the departure point to each node in the topology graph.

[0127] Calculate the difference between the two row vectors and select the preset norm type. This difference is measured. The norm type can be selected from a variety of available norms, such as the Frobenius norm, depending on the evaluation requirements. or norm Different norms vary in their sensitivity to differences in probability distributions and their measurement characteristics, and can be flexibly selected to adapt to different evaluation scenarios.

[0128] Step b4, set the key nodes The row differences corresponding to each node are determined by a preset aggregation function. By aggregating the data, we obtain the attacker's predicted state transition matrix error. .

[0129] Specifically, the set of key nodes The row difference metric corresponding to each node is aggregated using a preset function. By aggregating the results, we obtain the attacker's comprehensive predicted state transition matrix error. The calculation formula is as follows:

[0130]

[0131] in, The patroller's true state transition matrix The OK, Estimate the state transition matrix for the attacker The OK, For node indexing, For nodes The weight parameters, To select the top results after sorting by weight parameters in descending order A set of key nodes consisting of [number] nodes. Aggregate function. The maximum value can be obtained. Or take the arithmetic mean Choose from among them to suit different evaluation focuses: adopt During aggregation, the prediction error reflects the attacker's estimation bias towards the most difficult-to-predict critical nodes; [the text abruptly ends here, likely due to an incomplete sentence or missing information.] When aggregated, the prediction error reflects the attacker's average estimation bias for all critical nodes.

[0132] The attacker prediction error calculated through steps b1 to b4 above It can quantitatively reflect the ability of a patrol strategy to resist the attacker's predicted behavior in adversarial scenarios. The larger the prediction error, the greater the attacker's estimation deviation of the patroller's state transition matrix, meaning that the unpredictability of the patrol strategy plays a significant protective role in actual adversarial situations.

[0133] This embodiment achieves a refined quantitative evaluation of the protection effectiveness of patrol strategies through the aforementioned method for calculating the error of the attacker's predicted state transition matrix. This evaluation method has the following advantages: First, by introducing node weight parameters and a key node screening mechanism, the importance of nodes is directly integrated into the error calculation process, enabling the evaluation results to highlight the protection effectiveness of high-value, short-attack-time key nodes, avoiding evaluation biases that may result from equal weighting of all nodes in traditional evaluation methods. Second, it supports flexible configuration of multiple norm types and aggregation functions, allowing for the selection of appropriate measurement methods based on the evaluation needs of actual application scenarios, enhancing the versatility and adaptability of the evaluation framework. Third, by focusing the calculation of prediction error on the set of key nodes, it reduces computational complexity while ensuring the relevance of the evaluation, facilitating efficient execution in complex topology scenarios containing a large number of nodes. Fourth, the prediction error, as a quantitative indicator, can be compared with the aforementioned patrol strategy unpredictability index. PSUI The mutual corroboration forms a complete evaluation loop for patrol strategies, from theoretical optimization objectives to actual protection effects, providing an objective basis for iterative improvement of patrol strategies and horizontal comparison between different algorithms.

[0134] To further evaluate the protective effect of the patrol strategy of this invention in actual confrontation, the critical state mean relative error (KS-ARE) is calculated based on the attacker prediction model, and its formula is as follows:

[0135]

[0136] in, It is the patroller's true state transition matrix P of the th order. i OK; It is the attacker's prediction of the state transition matrix. The i OK; i It is a node index; It is a set of key nodes; It is the number of key nodes; determined by the aggregation function F. Take avg, Take F norm ,node Weight parameters The KS-ARE formula is derived; KS-ARE is the average relative error of the attacker's predictions for critical states, reflecting the "average" of the prediction model across all critical states. It smooths out the extreme errors of individual states, provides a measure of overall performance, and helps evaluate the stability and consistency of the prediction model. Experimental results are as follows: Figure 6 As shown.

[0137] Figure 6The horizontal axis k represents the attacker's observation sequence control parameter. A larger k value indicates that the attacker observes more patrol transitions. The vertical axis represents the attacker's average relative error in predicting critical states. As the attacker observes more patrol transitions, the error decreases, meaning the attacker's prediction becomes more accurate. The red SADEBO curve represents the curve generated by the simulated annealing dual-entropy balance optimization algorithm, the green DRAMSM curve represents the curve generated by the multivariate stochastic adaptive Markov steady-state matrix algorithm, and the blue MEDRO curve represents the curve generated by the multi-stage entropy-driven stochastic matrix optimization algorithm. From the figure, we can conclude that the SADEBO algorithm of this invention produces a higher curve than other curves under the same control parameter k, indicating that it is more difficult for the attacker to predict the patrol strategy generated by the patroller using the SADEBO algorithm.

[0138] To further highlight the protective effect on high-value critical nodes, the importance-weighted maximum row error (IW-MRE) is introduced as an evaluation indicator, and its formula is as follows:

[0139]

[0140] in, It is the patroller's true state transition matrix The i OK; It is the attacker's prediction of the state transition matrix. The i OK; i S is the node index; S is the set of key nodes; defined by the aggregation function. Take the maximum value. Take the 1-norm ,node Weight parameters Set as ,Right now The formula for IW-MRE is obtained. IW-MRE is the attack-weighted maximum row error based on the attack's prediction importance. It considers not only the magnitude of the prediction error but also the importance of the nodes, emphasizing the maximum error performance on important states. Experimental results are as follows: Figure 7 As shown.

[0141] Figure 7The horizontal axis k represents the attacker's observation sequence control parameter. A larger k value indicates that the attacker observes more patrol transitions. The vertical axis represents the importance-weighted maximum row error of the attacker's prediction. As the attacker observes more patrol transitions, the error decreases, meaning the attacker's prediction becomes more accurate. The red SADEBO curve represents the curve generated by the simulated annealing dual-entropy balance optimization algorithm, the green DRAMSM curve represents the curve generated by the multivariate stochastic adaptive Markov steady-state matrix algorithm, and the blue MEDRO curve represents the curve generated by the multi-stage entropy-driven stochastic matrix optimization algorithm. From the graph, we can conclude that the SADEBO algorithm of this invention produces a higher curve than other curves under the same control parameter k, indicating that it is more difficult for the attacker to predict the patrol strategy generated by the SADEBO algorithm. Figure 6 and Figure 7 This demonstrates that, regardless of the error index system, the patrol strategy generated by the SADEBO algorithm of this invention is not only more difficult to predict, but also has a certain degree of robustness.

[0142] Based on the above embodiments, as a specific implementation of the present invention, this embodiment further elaborates on the patrol strategy unpredictability index. The theoretical relationship between the method and the attacker's prediction error, and the specific application scenarios of the method of this invention.

[0143] one, Positive correlation with the attacker's prediction error.

[0144] For the steady-state distribution that satisfies the topological constraints and the target objective, generated by the simulated annealing bientropy balance optimization algorithm. The set of state transition matrices, with entropy weights within the policy. Entropy weight coefficient between strategies Under the condition that all are greater than zero, the patrol strategy unpredictability index It is positively correlated with the attacker's prediction error.

[0145] Specifically, when the set of strategies A higher value indicates a higher level of randomness in single-step transitions at the micro level and a higher level of diversity in mode switching at the macro level. In this case, it becomes more difficult for an attacker to estimate the patroller's state transition matrix by observing historical state sequences, and the deviation between the estimated result and the true state transition matrix also increases, manifesting as an increase in the attacker's prediction error. The rise. Conversely, if A lower value indicates a more regular patrol strategy, making it easier for attackers to learn the accurate transition probability distribution from the observed data, thus reducing the prediction error.

[0146] Based on the aforementioned positive correlation, in practical applications, it can be achieved by... This invention aims to indirectly improve the patrol strategy's resilience against attacker predictions by using it as an optimization objective. Simultaneously serving as both an evaluation criterion and an optimization objective for the strategy set, the simulated annealing dual-entropy balance optimization algorithm maximizes the desired outcome under constraints. The value enables the generated patrol strategy to achieve optimal unpredictability while satisfying the rationality of resource allocation. This effectively improves the unpredictability of the patrol path of the unmanned system and the prediction error of attackers, thereby enhancing the overall protection effect on critical infrastructure.

[0147] II. Application Scenarios of the Invention

[0148] The method of this invention can be widely applied to intelligent patrol and protection scenarios for various critical infrastructures. Specifically, the physical environments to be patrolled include, but are not limited to: port areas, where routine patrols and monitoring of key areas such as berths, cargo yards, and access channels are required; offshore energy platforms, such as offshore wind farms and oil drilling platforms, where continuous patrols and protection of platform facilities, surrounding waters, and pipeline areas are required; and border routes, such as border checkpoints, important passages, and areas along isolation facilities, where high-frequency and unpredictable patrols of sensitive areas are required.

[0149] In the above application scenarios, the method of the present invention can automatically generate differentiated target steady-state distributions based on the actual value and vulnerability of each facility node, and effectively deal with attackers with continuous observation capabilities through a spatiotemporally decoupled randomized patrol strategy, significantly improving the security protection level of critical infrastructure in adversarial environments.

[0150] In summary, the main technical effects of this invention are as follows:

[0151] (1) Integrating the degree of the node into the calculation of the steady-state distribution of the target makes the target distribution more consistent with the topological structure of the graph, and the algorithm is more likely to generate a matrix that satisfies the constraints.

[0152] (2) The strategy internal entropy of the patroller was designed to maintain the randomness of individual state transitions to the greatest extent, avoid deterministic patterns, and reflect the average randomness or uncertainty of the set of all state transition matrices.

[0153] (3) The inter-policy entropy of the patroller is designed to measure the degree of difference between different state transition matrices of the Markov chain, which helps to avoid the algorithm from converging to a solution with a single transition matrix and increasing the difficulty of prediction.

[0154] (4) A patrol strategy unpredictability index was designed to comprehensively reflect the uncertainty of the system at both the micro and macro levels, and to fully evaluate the randomness and diversity of the patrol strategy set. This index is a quantitative bridge between theoretical and actual patrol effects. The patrol strategy unpredictability index transforms the abstract concept of "unpredictability" into a measurable value. It serves as an evaluation standard for objective comparison of different algorithms and as an optimization target to guide algorithm design, thus achieving the unity of evaluation and optimization.

[0155] (5) The patrol strategy unpredictability index comprehensively quantifies the unpredictability of patrol strategies at different spatiotemporal scales: intra-strategy entropy measures the randomness of the transition probability within a single matrix, improving the unpredictability of patrol paths in a short period of time; inter-strategy entropy measures the overall difference between different matrices, ensuring the change of patrol patterns over a long period of time.

[0156] (6) A simulated annealing dual-entropy balance optimization algorithm for patrolling was developed. The algorithm takes maximizing the unpredictability index of patrolling strategy as its core objective and achieves this by simultaneously optimizing the intra-strategy entropy and inter-strategy entropy. During the optimization process, the algorithm needs to satisfy multiple conditions such as adjacency matrix constraints, transition probability row sum of 1 constraints, and target steady-state distribution constraints. The unpredictability index calculated by generating the patrol matrix using this algorithm is higher than the unpredictability index calculated by other algorithms.

[0157] (7) A counting-based estimator attacker algorithm was developed. This algorithm is completely data-driven and does not require any prior information assumptions. It only relies on the observed state trajectory data to learn the numerical properties of the state transition matrix.

[0158] (8) A multi-dimensional evaluation system based on critical states and node importance was designed, which provides a general method for evaluating the error of the attacker's predicted state transition matrix. Through the critical state screening mechanism, the importance of nodes is directly integrated into the error calculation, focusing on high-value nodes with short attack times, and measuring the error characteristics of different dimensions.

[0159] (9) The positive correlation between the patrol strategy unpredictability index and the attacker's prediction error was determined. Therefore, this index was used as the optimization target to guide the algorithm design. In practical applications, the patrol protection effect was improved by optimizing the patrol strategy unpredictability index.

Claims

1. A method for unmanned system patrol path decision-making in adversarial environment, characterized in that, include: Obtain an undirected patrol topology graph of the physical environment to be patrolled. Based on the value parameters, attack time parameters, and degree of each node in the topology graph, calculate the steady-state distribution of the target, including: obtaining the values ​​of each node in the patrol topology graph. Value parameters and attack time parameters ; Calculate each node Weight parameters The calculation formula is: Get each node Degree in the topological graph The degree For nodes Number of connected neighbor nodes; compute node Target steady-state distribution probability value The calculation formula is: in, n The target steady-state distribution represents the expected probability distribution of a patroller visiting each node during a long-term patrol, where the total number of nodes in the topology graph is denoted as . Define the unpredictability index of patrol strategies The patroller describes the transition probabilities between nodes in the patrol topology using a state transition matrix; It consists of a weighted sum of intra-strategy entropy and inter-strategy entropy; the intra-strategy entropy is used to measure the randomness of transitions within a single state transition matrix, and the inter-strategy entropy is used to measure the diversity of differences between different state transition matrices. The intra-strategy entropy is the arithmetic mean of the weighted average of the row entropies of each state transition matrix with the target steady-state distribution as the weight, and the inter-strategy entropy is the arithmetic mean of the average absolute difference between all pairs of different state transition matrices. A set of strategies is generated using a simulated annealing dual-entropy balance optimization algorithm, which aims to maximize the... To optimize the objective, while simultaneously satisfying the topological graph adjacency matrix constraint, the normalization constraint that the row sum of the transition probabilities is 1, and the objective steady-state distribution constraint, the solution space is iteratively searched and the state transition matrix is ​​updated according to the simulated annealing acceptance criterion, outputting a set of strategies containing multiple state transition matrices that satisfy the constraints. The unmanned system is controlled to perform a spatiotemporally decoupled patrol mission based on different state transition matrices in the strategy set. By randomly switching different state transition matrices and randomly transitioning within a single matrix according to probability, the patrol path of the unmanned system becomes unpredictable.

2. The method according to claim 1, characterized in that, The steps for calculating the internal entropy of the strategy include: For each state transition matrix in the policy set Q Calculate the first i row entropy The calculation formula is: Indicates from node Transfer to node The transition probability; Indicates from node Departure to Node The uncertainty of the transition probability; With target steady-state distribution Using the row entropy as the weight, a weighted sum of the row entropies is obtained to obtain the matrix. P Weighted average row entropy The calculation formula is: Used to characterize the uncertainty of a single state transition matrix; For all matrices The arithmetic mean is used to obtain the internal entropy of the strategy. The formula is: in The size of the strategy set.

3. The method according to claim 1, characterized in that, The steps for calculating the inter-strategy entropy include: Calculate the two state transition matrices in the policy set Q. and The average absolute difference between The calculation formula is: in, For the first in set Q s A state transition matrix; For the first in set Q t A state transition matrix; The larger the value, the greater the difference in probability distribution between the two state transition matrices; For the first s A state transition matrix from node Transfer to node The probability of; For the first t A state transition matrix from node Transfer to node The probability of; Compute all distinct matrix pairs in the strategy set Q The arithmetic mean of the strategies is used to obtain the inter-strategy entropy. The calculation formula is: in, It is the reciprocal of the number of all state transition matrix pairs; This is used to measure the average difference between all matrix pairs in the policy set Q. The larger the average difference, the more diverse the policy set generated by the simulated annealing dual-entropy balance optimization algorithm is in terms of transition probability patterns, and the more difficult it is for an attacker to predict.

4. The method according to claim 1, characterized in that, The unpredictability index of the patrol strategy The calculation formula is: in, It is the internal entropy of the strategy; It is the entropy between strategies; It is the weighting coefficient of the strategy's internal entropy; These are the weighting coefficients of the inter-strategy entropy; the aforementioned This is a comprehensive evaluation index of the quality of the strategy set, which comprehensively reflects the uncertainty of the strategy set in two dimensions: transfer randomness and difference diversity; The higher the score, the stronger the unpredictability of the strategy set.

5. The method according to claim 1, characterized in that, The strategy set generated using the simulated annealing dual-entropy balance optimization algorithm includes: Input the topological adjacency matrix of the Markov chain, the target steady-state distribution, the random seed, and the number of target matrices to be generated; In the initialization phase, the number of target matrices and the initial state transition matrix are generated; the generation process includes: randomly generating self-loop probabilities for each node, constructing off-diagonal elements using detailed balance conditions, and normalizing all rows to ensure that the generated matrix is ​​a Markov transition matrix. In the simulated annealing optimization phase, after initializing the annealing simulation parameters, the following iterative process is executed: randomly selecting matrix elements in the current state transition matrix for adjustment, while maintaining the normalization constraint that the row sum is 1 after adjustment; calculating the actual steady-state distribution corresponding to the adjusted matrix, and calculating the error between it and the target steady-state distribution; to maximize the target steady-state distribution. To optimize the objective, the algorithm decides whether to accept the current adjustment based on the simulated annealing acceptance criterion; the iteration is repeated until the algorithm converges to the target steady-state distribution or reaches the preset maximum number of iterations. Output a set of state transition matrices that satisfy the adjacency matrix constraint, the row sum of 1 constraint, and the target steady-state distribution constraint.

6. The method according to claim 1, characterized in that, The method also includes an evaluation step for predicting attacker behavior: The attacker estimates the patroller's state transition matrix using a counting-based estimator algorithm, specifically including: The input to the counting-based estimator algorithm is the observed state sequence. and number of states The output is the estimated state transition matrix. ; Statistical analysis of each transition in the state sequence Number of times, Indicates from state to state The number of transfers; For each state Calculate the total number of transitions starting from this state. ,in, For the first The sum of rows; For each state pair The estimated transition probability is ; If a certain state never appears as a starting state in the observation sequence, then the transition probability of that state is set to a uniform distribution.

7. The method according to claim 6, characterized in that, The evaluation step also includes calculating the error of the attacker's predicted state transition matrix: The true state transition matrix of the patroller is determined as follows: The attacker estimates the state transition matrix as follows: ; The weight parameters of each node are determined based on its value parameters and attack time parameters. All nodes are then sorted in descending order according to these weight parameters, and the top nodes are selected. These nodes constitute the key node set. ; Calculate the true state transition matrix The Rows and estimation of state transition matrix The Differences between rows, and select a preset norm type. Measure this difference; The set of key nodes The row differences corresponding to each node are determined by a preset aggregation function. By aggregating the data, we obtain the attacker's predicted state transition matrix error. The calculation formula is as follows: in, It is the patroller's true state transition matrix The OK; It is the attacker's prediction of the state transition matrix. The OK; It is a node index. It is a node Weight parameters; It is a set of key nodes, which sort all states according to... Arrange the states in descending order and select the first m states to form the key node set; It is an aggregate function. It is a norm type.

8. The method according to claim 1, characterized in that, For a given topological constraint and target steady-state distribution The set of transition matrices, with entropy weight coefficients within the strategy. Entropy weight coefficient between strategies Under the condition that all are greater than zero, the It is positively correlated with the attacker's prediction error; by the The optimization goal is to improve the unpredictability of unmanned system patrol paths and the prediction error of attackers, and to enhance the overall protection of critical infrastructure. The physical environment to be patrolled includes critical infrastructure areas in ports, offshore energy platforms, or border crossings.

9. The method according to any one of claims 1 to 8, characterized in that, The unpredictability index of the patrol strategy The evaluation criterion for the strategy set is also the optimization objective of the simulated annealing dual-entropy balance optimization algorithm, so as to achieve the unification of the evaluation and optimization of patrol strategies.

Images