An estimation method for counteracting an attacker's estimation of an unmanned system's patrol strategy in an environment

Abstract

This invention discloses a method for estimating the patrol strategy of an unmanned system by an attacker in an adversarial environment, belonging to the field of unmanned system patrol strategy analysis and estimation technology. It includes: the attacker continuously observes the movement trajectory of the patroller and records the state sequence; based on the state sequence, a maximum likelihood estimator for transition frequency is used as the basic estimation method, by statistically analyzing the frequency of transitions between states, constructing a transition count matrix, and normalizing it to estimate the state transition matrix from a state sequence; and on top of the maximum likelihood estimator framework, an adaptive time decoupling ensemble estimator is employed, introducing a time interval parameter. t Temporal decoupling of the state sequence and adaptive generation of candidate data based on data size. t value set, through K Cross-validation to select the optimal t The values ​​are weighted and integrated according to the scoring weights to obtain the final state transition matrix estimation result. This invention provides a purely data-driven patrol policy estimation method from the attacker's perspective.

Description

Technical Field

[0001] This invention relates to the field of analysis and estimation technology of patrol strategies for unmanned systems, and particularly to a method for estimating the patrol strategies of unmanned systems by attackers in adversarial environments. Background Technology

[0002] Autonomous unmanned system patrol technology has become an important means of protecting critical infrastructure due to its continuous monitoring and rapid response capabilities. In practical deployments, the patrol path decision of unmanned systems is usually characterized by a Markov chain state transition matrix, in which the transition probabilities of the patroller between each patrol node are fully encoded. Since patrol behavior is inherently governed by probabilistic strategies and has observable and modelable intrinsic regularities, attackers may be able to infer patrol strategies by observing patrol behavior over a long period and analyzing its statistical patterns.

[0003] However, existing research largely focuses on the patroller's perspective, concentrating on generating highly randomized patrol strategies to enhance defense effectiveness, while lacking research on modeling and estimating patrol strategies from the attacker's perspective. In real-world adversarial scenarios, attackers can approximate the true distribution of the patroller's state transition matrix using statistical learning techniques, even without knowing the patroller's actual state transition matrix, solely through continuously observed and accumulated state sequence data. Therefore, how to accurately estimate the state transition matrix of unmanned system patrol strategies from the attacker's perspective under purely data-driven conditions is a crucial technical problem that urgently needs to be solved to improve the theoretical framework of adversarial games and enhance the robustness of defense strategies. Summary of the Invention

[0004] The purpose of this invention is to provide a method for estimating the patrol strategy of unmanned systems by attackers in adversarial environments, addressing the lack of modeling and estimation of the behavioral patterns of unmanned patrol systems from the attacker's perspective in existing research, and achieving high-precision estimation of the patroller's state transition matrix under purely data-driven conditions. The method includes: Attackers continuously observe the patrollers' movement trajectories and record their state sequences; Based on the state sequence, the attacker uses the transition frequency maximum likelihood estimator as the basic estimation method. The transition frequency maximum likelihood estimator estimates the state transition matrix from a state sequence by counting the frequency of transitions between states, constructing a transition count matrix and normalizing it. Building upon the framework of the aforementioned maximum likelihood estimator for transition frequencies, the attacker employs an adaptive time-decoupled ensemble estimator to perform the following steps: Based on the length of the state sequence, candidate time interval parameters are adaptively generated. A set; pass K Cross-validation, for each candidate Performance evaluation of the values: for each candidate Value, sort the state sequence according to K The dataset is divided into a training set and a validation set using a folding method. On the training set, the data is then processed according to... The state transition matrix is ​​estimated by partitioning the subset and calling the transition frequency maximum likelihood estimator. The negative log-likelihood score is then calculated on the validation set using this state transition matrix, and the result is obtained by averaging across all folds. The overall performance score is calculated, and a preset number of items with the highest overall performance scores are selected. value as optimal value; For each optimal The value is calculated using a normalized weight based on its overall performance score; For each optimal The value is used to divide the entire state sequence into modulo operations. Each subset is evaluated using the maximum likelihood estimator for the transition frequency, and the optimal values ​​are obtained by averaging the results. The state transition matrix corresponding to the value; The state transition matrices are weighted and averaged according to the normalized weights to obtain the final state transition matrix estimation result. Based on the final state transition matrix estimation result, the attacker infers the patroller's behavior patterns, identifies patrol blind spots, and determines the optimal attack window.

[0005] Optionally, the transition frequency maximum likelihood estimator estimates the state transition matrix from a state sequence by statistically analyzing the frequency of transitions between states, constructing a transition count matrix, and normalizing it, including: The state sequence is represented as follows: ,in T Let be the sequence length, and let 'a' be the number of states of the patroller. For each transition in the aforementioned state sequence, increment the corresponding element of the counting matrix by 1 to obtain an a×a counting matrix, where each element... Indicates from state to state The number of transfers; For each state Calculate the total number of transitions starting from this state. ,in, For the counting matrix, the first The sum of rows; For each state pair Estimated state transition probabilities for: ; when When this occurs, the state transition probability of that state is set to a uniform distribution. Representing state It never appears as the starting state in the aforementioned state sequence.

[0006] Optionally, the method of... K Cross-validation, for each candidate The values ​​are used for performance evaluation, including: The state sequence is represented as a transition pair sequence. and in accordance with K The training set is divided into training and validation sets using folded cross-validation; for each fold of the training set, the current candidate... Values ​​divide it into There are subsets, and the time interval between adjacent transition pairs within each subset is... ; For the patrolmen in the The state at any given moment. For the patrolmen in the The state at any given moment. T The sequence length; On the training set, the transition frequency maximum likelihood estimator is called separately for each subset to estimate the sub-transition matrix. For a given The value is averaged over the subtransition matrices of all non-empty subsets to obtain the comprehensive estimation matrix on the current training set: ; in, This is the current cross-validation step number; Number the subsets ∈{0,1,..., -1}, total A subset; The set of indices for non-empty subsets; Indicates the first The training set of the first fold Individuals clustered together, patrolmen from the state Transition to state The actual number of observations; Indicates the first The training set of the first fold Individuals clustered together, patrolmen from the state Total number of transfers from the starting point; Indicates the first The training set of the first fold On a subset, the transition frequency maximum likelihood estimator estimates the states from... Transition to state The probability value; Indicates the first On the training set, use When the value is zero, the comprehensive estimated transition probability is obtained by averaging the estimated matrices of all non-empty subsets. On the corresponding validation set, the negative log-likelihood of the comprehensive estimation matrix is ​​used as the basis for the validation. The performance score at this discount: ; in, Indicates the first The total number of sample transition pairs in the validation set; This indicates traversing each actually observed state transition in the verification set; For the first Used in cross-validation The negative log-likelihood value; Regarding Value in all K The average of the performance scores is used to obtain the overall performance score. : ; in, K This is the total number of folds for cross-validation, and its value is dynamically adjusted based on the sequence length of the state sequence.

[0007] Optionally, the preset number of samples with the highest overall performance scores are selected. value as optimal Values, including: From all candidates In the values, select the comprehensive performance score. The highest front indivual Values ​​constitute the optimal Value set: ; in, For the preset optimal Number of values as a candidate Any subset of values, The optimal selection Value set.

[0008] Optionally, the optimal pair The value is calculated with a normalized weight based on its overall performance score, including: For the optimal Each in the set of values value Calculate its normalized weights using the following formula: ; in, For the first The best The overall performance score is worthwhile. For all optimal The sum of the performance scores of the values for The corresponding normalized weights.

[0009] Optionally, the optimal pair The value is used to divide the entire state sequence into modulo operations. Each subset is evaluated by calling the maximum likelihood estimator of the transition frequency on each subset, and the optimal values ​​are obtained by averaging. The state transition matrix corresponding to the value includes: For the optimal Each optimal value in the set value , make the sequence length of T The complete state sequence is divided into modulo operations. Subset: ; in, Indicates time t Divide by The remainder is used to evenly distribute the complete state sequence to A subset; The first division based on the modulo operation A subset; For each subset, the transition frequency maximum likelihood estimator is called independently to estimate the state transition matrix, and then... The optimal state transition matrix is ​​obtained by averaging the state transition matrices of the subsets. The state transition matrix corresponding to the value .

[0010] Optionally, the step of weighting each state transition matrix according to the normalized weights to obtain the final state transition matrix estimation result includes: ; in, The final state transition matrix estimation result output by the adaptive time decoupling ensemble estimator is the core basis for attackers to infer the patroller's behavior patterns and identify patrol blind spots.

[0011] Optionally, the step of adaptively generating candidate time interval parameters based on the length of the state sequence... The set includes: Determine the basic candidate value ∈{2,3,4,5,6,7}; When the length of the state sequence T When the value is greater than 100, add {8, 10, 12}; When the length of the state sequence T When the value is greater than 200, add {15, 18, 20}; When the length of the state sequence T If the value is greater than 500, further increase {25,30}.

[0012] Optionally, the patroller is an autonomous unmanned system performing patrol missions, including drones, unmanned vehicles, or unmanned boats / vessels; the attacker is an adversary equipped with sensing devices and computing units, wherein the computing units are used to perform data processing steps of the estimation method, and the sensing devices are used to continuously observe the movement trajectory of the patroller.

[0013] Optionally, embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method provided in the embodiments of the present invention.

[0014] The technical solution provided by this invention offers a method for estimating the state transition matrix of an unmanned system patrol strategy from an attacker's perspective, filling the technical gap in attacker-perspective modeling and analysis in existing research. This method is entirely data-driven. Under the condition that the attacker has no knowledge of the patroller's true state transition matrix and cannot predict in advance whether it has special structures such as low rank, the method relies solely on state sequence data accumulated through continuous observation. It uses a maximum likelihood estimator to count the frequency of transitions between states, constructs a transition count matrix, and normalizes it to obtain a preliminary estimate. Then, an adaptive time-decoupled ensemble estimator is employed, introducing a time interval parameter. Temporal decoupling of the state sequence, dividing temporally coupled samples into approximately independent subsequences, fundamentally eliminates the interference of inherent temporal correlation in Markov chain observation sequences on frequency statistics, significantly improving the estimation accuracy of the state transition matrix. Furthermore, this method can adaptively adjust candidate sequences based on the scale of the observed data. The search range for values ​​is automatically filtered for the optimal value through cross-validation. The values ​​are weighted and integrated according to performance weights, without the need for manual parameter specification. It has good generalization ability and practicality, and provides reliable data support for attackers to infer patrol behavior patterns, identify patrol blind spots and select the optimal attack window. Attached Figure Description

[0015] Figure 1 A flowchart illustrating an attacker's estimation method for patrol strategies of unmanned systems in an adversarial environment, provided as an embodiment of the present invention; Figure 2A schematic diagram of a Markov chain topology containing five nodes provided in an embodiment of the present invention; Figure 3 A schematic diagram of the patrol strategy state transition matrix provided in an embodiment of the present invention; Figure 4 This diagram illustrates the comparison of estimation errors between the Transfer Frequency Maximum Likelihood Estimator (TFMLE) and the Adaptive Time Decoupling Integrated Estimator (ATDE) at different observation scales. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0017] This invention provides a method for an attacker to estimate the patrol strategy of an unmanned system in an adversarial environment. The attacker continuously observes the patroller's movement trajectory, records the patroller's state sequence, and estimates the patroller's state transition matrix based on the state sequence.

[0018] Specifically, the physical environment of the patroller is first discretized and modeled using an undirected topological graph of Markov chains. The patroller moves randomly on the topological graph with probability, and its patrol strategy is fully described by the state transition matrix. The attacker has no idea about the patroller's true state transition matrix and can only estimate it by continuously observing and accumulating state sequence data.

[0019] The estimation method of this invention comprises two progressive levels: The first layer is a transition frequency maximum likelihood estimator. Based on the maximum likelihood estimation principle, the attacker constructs a transition count matrix by statistically analyzing the frequency of each state transition in the state sequence and then normalizes it to obtain a preliminary estimate of the patroller's state transition matrix. This method is a fundamental scheme for data-driven estimation, simple to implement and computationally efficient. However, due to the inherent temporal correlation in the Markov chain state transition sequence, the estimation accuracy of this method is somewhat limited.

[0020] The second level is an adaptive time-decoupled ensemble estimator. This estimator, based on the framework of the transition frequency maximum likelihood estimator, introduces a time interval parameter. Temporal decoupling of state sequences involves segmenting temporally coupled samples into approximately independent subsequences, fundamentally eliminating the interference of temporal correlation on frequency statistics. Its core steps include: an adaptive generation mechanism for time interval parameters based on data scale; and... KThe adaptive time-decoupled ensemble estimator employs the following steps: performance evaluation of candidate parameters for cross-validation; selection of the optimal time interval parameter set based on the log-likelihood criterion; normalized allocation of ensemble weights based on cross-validation performance; temporal decoupling re-estimation of the full observation data based on modulo-segmentation; and weighted ensemble fusion of multi-scale transition matrices based on normalized weights. Through these steps, the adaptive time-decoupled ensemble estimator effectively reduces the estimation bias introduced by temporal correlation and significantly improves the estimation accuracy of the state transition matrix.

[0021] Based on the final state transition matrix estimation results, attackers can infer the patroller's behavior patterns, identify the timing and location of patrol blind spots, and provide decision support for selecting the optimal attack window.

[0022] The technical solution of the present invention will be further described in detail below with reference to specific steps.

[0023] like Figure 1 As shown in the embodiment of the present invention, a method for estimating an attacker's patrol strategy for an unmanned system in an adversarial environment may include the following steps: S110, the attacker continuously observes the patroller's movement trajectory and records the patroller's state sequence.

[0024] In this embodiment, "patrolman" refers to any autonomous unmanned system performing patrol missions, including but not limited to drones, unmanned vehicles, and unmanned boats / vessels. The physical environment of the patrolman is discretized and represented by a Markov chain undirected topological graph. To perform modeling, among which Each node in the graph represents a set of patrol locations. This represents locations in a real-world environment where unmanned systems need to be patrolled and monitored. n Indicates the total number of patrol positions; Describe the set of edges. >0 represents a node Transfer to node The probability satisfies Among them, the edge set and The index of the node, and the state described in the subsequent state transition. and state The correspondence is consistent. For example... Figure 2 This represents a Markov chain topology graph containing five nodes.

[0025] The patroller moves on the undirected topology graph of the Markov chain patrol described above, and its behavior decision is determined by the state transition matrix. P Full description, matrix P elements Indicates that the patrolman starts from the node Transfer to node The probability, the patrol strategy of the patroller is fully encoded in P In the midst of it. At every moment, the patrolmen, based on... P The probability distribution of the corresponding row randomly selects the next patrol node, thus forming a Markov chain trajectory on the graph, such as... Figure 3 The diagram shown is a schematic of the patrol strategy state transition matrix for the patroller.

[0026] In this embodiment, "attacker" refers to an adversary that infiltrates or attacks key targets within the patrol area. The attacker is an adversary equipped with sensing devices and computing units. The computing unit is used to execute the data processing steps of the estimation method proposed in this embodiment of the invention, and the sensing devices are used to continuously observe the movement trajectory of the patroller.

[0027] Attacker's true state transition matrix for the patroller P Completely unknown, and it is impossible to determine in advance whether the matrix has a low-rank or other special structure. The only usable source of information is the patrol state sequence data accumulated through continuous observation. By continuously observing the movement trajectory of the patrollers, the attacker records their historical state sequence, which is the only data basis for the attacker to make subsequent estimations.

[0028] S120, the attacker uses the transition frequency maximum likelihood estimator as the basic estimation method based on the state sequence. The transition frequency maximum likelihood estimator estimates the state transition matrix from a state sequence by counting the frequency of transitions between states, constructing a transition count matrix and normalizing it.

[0029] Specifically, the transition frequency maximum likelihood estimator is a fundamental method for attackers to estimate the state transition matrix of an unmanned system's patrol strategy. Its working principle is based on the maximum likelihood estimation principle. The algorithm's input is a sequence of states. Given the number of states 'a', the output is the estimated state transition matrix.

[0030] Specifically, the attacker first counts the number of transitions: for each transition in the state sequence... , the corresponding counting matrix elements Adding 1 results in a counting matrix, where each element... Indicates from state to state The number of transitions. Then calculate the total number of departures for each state: for each state Calculate the total number of transitions starting from this state. That is, the counting matrix of the first... The sum of rows. Finally, estimate the transition probabilities: for each state pair Estimated transition probability for If a state never appears as a starting state in the observation sequence, i.e. If the transition probability of this state is set to a uniform distribution, then the transition probability of this state is set to a uniform distribution.

[0031] This method is simple to implement and computationally efficient, serving as a fundamental approach for data-driven estimation. However, due to the inherent temporal correlation in the state transition sequences of Markov chains, the estimation accuracy of this method is somewhat limited.

[0032] S130, on top of the framework of the transition frequency maximum likelihood estimator, the attacker uses an adaptive time-decoupled ensemble estimator to perform the following steps S140 to S180.

[0033] Specifically, the adaptive time-decoupled ensemble estimator inherits the frequency statistics framework of the transition frequency maximum likelihood estimator, and introduces a time interval parameter on this basis. The observed sequence is segmented, decoupling temporally coupled samples into approximately independent subsequences, fundamentally eliminating the interference of temporal correlation on frequency estimation. The specific implementation steps include S140 to S180.

[0034] S140, Adaptively generate candidate time interval parameters based on the length of the state sequence. A set of.

[0035] Attackers obtain a sequence of states of length by continuously observing the patroller's state sequence. T The observed trajectory data. Based on the scale of the observed data. T Adaptive dynamic adjustment of candidates Search range of values: basic candidate values ∈{2,3,4,5,6,7}; when T When >100, add {8,10,12}; when T When the value is greater than 200, add {15, 18, 20}; when... T When the value is >500, further increase the number of candidates to {25,30}. The larger the scale of the observed data, the more candidates... The wider the search range of values, the more likely it is to ensure that the most suitable time interval parameter for the current data can be found under different observation conditions.

[0036] Time interval parameter It is a key parameter that determines how to segment the Markov chain trajectory into approximately independent sample sets. It represents both the time interval between adjacent transition pairs within the same subset and the number of subsets after dividing the original state sequence. As a sampling interval, it controls the time distance between samples, affecting the correlation between samples; as the number of subsets, it controls the granularity of data partitioning, affecting the number of samples in each subset.

[0037] S150, via K Cross-validation, for each candidate Performance evaluation of the values: for each candidate Value, sort the state sequence by K The dataset is divided into a training set and a validation set using a folding method. On the training set, the data is then processed according to... The state transition matrix is ​​estimated by partitioning the subset and calling the transition frequency maximum likelihood estimator. The negative log-likelihood score is then calculated on the validation set using this state transition matrix, and the result is obtained by averaging across all folds. The overall performance score is calculated, and a preset number of items with the highest overall performance scores are selected. value as optimal value.

[0038] Specifically, cross-validation uses the KFold class, with a fold count of... K Adjust dynamically based on the amount of data. For each candidate... The value is determined by the following process: for each fold of the validation process, the transition probabilities are learned on the transition pairs in the training set, and then the log-likelihood of this estimated matrix is ​​calculated on the validation set. This value is used to measure the accuracy of the matrix's predictions on the validation data, and the results are evaluated one by one. Value, for each candidate Each value completes the following process independently, without affecting the others: (1) Data segmentation: The attacker will represent the state sequence as a sequence of transition pairs. and in accordance with K The split is performed using cross-validation. Each fold contains a training set and a validation set. For each fold's training set, further processing is performed according to the current candidate... Values ​​divide it into There are subsets, and the time interval between adjacent transition pairs within each subset is... This achieves decoupling of time-related factors.

[0039] (2) Model training: On the training set, the transition frequency maximum likelihood estimator is called independently to estimate the transition matrix for each subset. By averaging the estimated matrices of all non-empty subsets, the comprehensive estimated matrix on the current training set is obtained.

[0040] (3) Model Evaluation: Calculate the log-likelihood score of the comprehensive estimation matrix on the corresponding validation set, and evaluate the performance of this τ value on the validation set. The smaller the negative log-likelihood value, the closer the estimated matrix is ​​to the patroller's true state transition matrix. The more reasonable the value selection, the better.

[0041] (4) Average performance: for this Value in all K The average of the performance scores is used to obtain the overall performance score. This will be used as a basis for selecting the best option. The basis for the value.

[0042] After completing the above evaluation, based on the comprehensive performance score, candidates will be selected from all candidates. Select the preset number of values ​​with the highest scores. Values ​​constitute the optimal The set of values ​​is used for subsequent weighted ensemble estimation.

[0043] To ensure a complete and clear description of the solution, the specific implementation of S150 will be explained in detail in the following embodiments.

[0044] S160, for each optimal The value is calculated using a normalized weight based on its overall performance score.

[0045] For the optimal Each in the set of values Value, scored according to its overall performance. Calculate the normalized weights. Higher performance scores indicate better performance. The higher the weight assigned to a value, the more it reflects the various... The contribution ratio of the value to the final integrated estimate, with the sum of all weights being 1.

[0046] S170, for each optimal The value is used to divide the entire state sequence into modulo operations. Each subset is evaluated using the maximum likelihood estimator for the transition frequency, and the optimal values ​​are obtained by averaging the results. The state transition matrix corresponding to the value.

[0047] It should be noted that the purpose of cross-validation in S150 is solely for evaluation and screening. The value is not used in the final matrix estimation. In this step, the optimal... Each optimal value in the set value The attacker re-estimates using all of their observation data to make full use of all available observation information.

[0048] Specifically, for the optimal Each optimal value in the set value , with a length of T The complete state sequence is divided into modulo operations. Subset: .

[0049] in, For the patrolmen in the The state at any given moment. For the patrolmen in the The state that a person is transitioning to at any given moment; Indicates time t Divide by The remainder is used to evenly distribute the complete sequence to... A subset. The first division based on the modulo operation A subset containing all that satisfy The transition pairs, the time interval between adjacent samples within the subset is Step, approximately independent; for each subset, independently call the transition frequency maximum likelihood estimator to estimate the transition matrix, and then... The average of the estimated matrices of each subset is used to obtain the... The final estimated matrix corresponding to the value ,Right now For attackers to use complete observation data and the first The best The transition matrix obtained by value estimation.

[0050] S180, the state transition matrices are weighted and averaged according to the normalized weights to obtain the final state transition matrix estimation result.

[0051] The multiple estimated matrices obtained in S170 are normalized according to the weights calculated in S160. By performing a weighted average and fusing the estimation results from different time scales, the final state transition matrix estimation result output by the adaptive time-decoupled ensemble estimator is obtained. .

[0052] S190: Based on the final state transition matrix estimation results, the attacker infers the patroller's behavior patterns, identifies patrol blind spots, and determines the optimal attack window.

[0053] Based on the final state transition matrix estimation results, attackers infer the patroller's behavior patterns, identify the timing and location of patrol blind spots, and provide decision support for selecting the optimal attack window, thereby completing the infiltration or strike mission with minimal exposure risk within the optimal time window.

[0054] The technical solution provided by this invention offers a method for estimating the state transition matrix of an unmanned system patrol strategy from an attacker's perspective, filling the technical gap in attacker-perspective modeling and analysis in existing research. This method is entirely data-driven. Under the condition that the attacker has no knowledge of the patroller's true state transition matrix and cannot predict in advance whether it has special structures such as low rank, the method relies solely on state sequence data accumulated through continuous observation. It uses a maximum likelihood estimator to count the frequency of transitions between states, constructs a transition count matrix, and normalizes it to obtain a preliminary estimate. Then, an adaptive time-decoupled ensemble estimator is employed, introducing a time interval parameter. Temporal decoupling of the state sequence, dividing temporally coupled samples into approximately independent subsequences, fundamentally eliminates the interference of inherent temporal correlation in Markov chain observation sequences on frequency statistics, significantly improving the estimation accuracy of the state transition matrix. Furthermore, this method can adaptively adjust candidate sequences based on the scale of the observed data. The search range for values ​​is automatically filtered for the optimal value through cross-validation. The values ​​are weighted and integrated according to performance weights, without the need for manual parameter specification. It has good generalization ability and practicality, and provides reliable data support for attackers to infer patrol behavior patterns, identify patrol blind spots and select the optimal attack window.

[0055] exist Figure 1 Based on the illustrated embodiment, as one implementation of the present invention, in S150, through... K Cross-validation, for each candidate τ Performance evaluation of values ​​can include the following steps: Step a1, represent the state sequence as a sequence of transition pairs. and in accordance with K The training set is split into training and validation sets using folded cross-validation. For each fold of the training set, the current candidate... Values ​​divide it into There are subsets, and the time interval between adjacent transition pairs within each subset is... . For the patrolmen in the The state at any given moment. For the patrolmen in the The state at any given moment.

[0056] Specifically, the attacker will use the original transition pair sequence obtained through observation to... K The split is performed using cross-validation. Folding includes training set and verification set For each fold of the training set, further refine the selection based on the current candidate... Values ​​divide it into There are subsets, and the time interval between adjacent transition pairs within each subset is... This achieves decoupling of time-related factors.

[0057] in, T This represents the total length of the state sequence accumulated by the attacker through continuous observation of the patroller, that is, the total number of patroller state transitions recorded by the attacker. This indicates the current cross-validation step number. Indicates the first A compromise is made using a subset of transition pairs for training, which the attacker uses to estimate the transition matrix. Indicates the first A compromise is made for the subset of transition pairs used for verification, which the attacker uses to evaluate the estimated effect. This represents the time interval parameter, which controls the time distance between adjacent transition pairs within a subset, and is also equal to the number of subsets.

[0058] Cross-validation uses the KFold class, with a fold count of... K Adjust dynamically based on the amount of data. For each candidate... The value is determined by the fact that each fold of the validation process involves learning transition probabilities on the transition pairs in the training set, and then calculating the log-likelihood of this estimated matrix on the validation set. This value measures the accuracy of the matrix's predictions on the validation data. Evaluation is performed one by one. Value, for each candidate Each value completes the following process independently without affecting the others.

[0059] Step a2, in the training set For each subset, the transition frequency maximum likelihood estimator is called to estimate the transition matrix, thus obtaining the sub-transition matrix. For a given The value is averaged over the subtransition matrices of all non-empty subsets to obtain the comprehensive estimation matrix on the current training set: ; in, This is the current cross-validation step number. Number the subsets ∈{0,1,..., -1}, total A subset; It is the set of indices for non-empty subsets, that is, the set of subset numbers that actually contain transition samples in the state sequence.

[0060] Indicates the first The training set of the first fold Individuals clustered together, patrolmen from the state Transition to state The actual number of observations.

[0061] Indicates the first The training set of the first fold Individuals clustered together, patrolmen from the state Total number of transfers from the starting point.

[0062] Indicates the first The training set of the first fold On a subset, the transition frequency maximum likelihood estimator estimates the states from... Transition to state The probability value.

[0063] Indicates the first On the training set, use When the value is given, the comprehensive estimated transition probability obtained by averaging the estimation matrices of all non-empty subsets represents the transition probability from state. Transition to state The probability is used to measure the predictive power of the estimated matrix for actual transitions in the validation set.

[0064] Step a3, in the corresponding validation set Above, the negative log-likelihood of the comprehensive estimation matrix is ​​used as the basis for this. The performance score at this discount: .

[0065] in, Indicates the first The total number of sample transition pairs in the folded validation set is used for normalization to eliminate the influence of different folded validation set sizes on the scoring. This indicates that each observed state transition in the verification set is traversed. This is the initial state of the transition. The target state for the transition. For the first Used in cross-validation The negative log-likelihood value is a performance metric used to measure the estimation matrix's performance. A smaller value indicates that the estimated matrix is ​​closer to the patroller's true state transition matrix. The more reasonable the value selection, the better.

[0066] Step a4, regarding the Value in all K The average of the performance scores is used to obtain the overall performance score. .

[0067] As can be seen from step a3 above, for a certain candidate The value is such that each completed cross-validation step generates a corresponding negative log-likelihood score. Therefore, in K After all cross-validation is completed, each candidate Value accumulation K Each candidate has a validation score. To eliminate the impact of randomness in the partitioning of single-fold data on the scoring, a validation score is calculated for each candidate. The value will be in all K The average of the negative log-likelihood scores on the fold is used to obtain the result. The final overall performance score is as follows: .

[0068] in, This represents the candidate time interval parameter currently being evaluated. K This represents the total number of folds in cross-validation, used for... K The average of the validation scores is used to eliminate the influence of randomness in a single data split. Indicates the fold number. ∈{1,2,…, K}, iterate through all K fold. Indicates the first Used in cross-validation The negative log-likelihood value at the given value is calculated in step a3. Indicates candidate Value in all K The average performance score on the discount The more accurate the overall estimate of the patrolman's state transition matrix, the more likely it is to be used as the optimal value for subsequent selection. The basis for the value.

[0069] This embodiment is illustrated by... K Cross-validation for each candidate Performance evaluation is performed by splitting the state sequence into a training set and a validation set, and then performing a performance evaluation on the training set according to... The values ​​are partitioned into subsets, and the transition frequency maximum likelihood estimator is called separately for each subset. The average of these estimates is then calculated, and the negative log-likelihood on the validation set is used as the performance score. The average across all folds is then used to obtain the comprehensive performance score, which can objectively and stably measure the performance of each candidate. τ The impact of the value on the accuracy of state transition matrix estimation is used to select the optimal value in subsequent steps. The value provides a reliable basis.

[0070] Based on the above K The overall performance score obtained by evaluating the candidate parameters one by one in the cross-validation process Comparing different sample interval parameters The log-likelihood of the model generated on the validation set is used to select the parameter settings that best capture the dynamic characteristics of the Markov chain, i.e., the top-scoring parameters. indivual value as optimal Value set: ; in, For the preset optimal The number of values, which participate in weighted fusion during control ensemble estimation. Quantity of values; as a candidate Any subset of values ​​from all candidates In the values, select the comprehensive performance score. The highest front indivual Values ​​constitute the optimal Value set. The optimal selection A set of values ​​containing The time interval parameter that performs best on the validation set.

[0071] This embodiment uses comprehensive performance scoring as an objective basis, selecting from all candidates. The system automatically selects the preset number of values ​​with the highest scores. Optimal value composition Value set, avoiding manual specification The subjectivity and blindness of the value selection ensure that the chosen... The value exhibits the best predictive performance on the validation set, providing a reliable parameter basis for subsequent weighted ensemble estimation.

[0072] Based on the above embodiments, as one implementation of the present invention, the optimal... The value, calculated with a normalized weight based on its overall performance score, may include the following steps: The optimal ones selected above Each in the set of values value Calculate its normalized weights using the following formula. The weights are proportional to its cross-validation score; the higher the score, the lower the weight. The larger the value, the greater the weight it is assigned. ; in, For the first The best The overall performance score is based on the above. K The candidate parameters for cross-validation are calculated through a step-by-step performance evaluation process, and are used as the molecular representation. The relative advantage of value.

[0073] For all optimal The sum of the performance scores is used as the denominator to normalize the weights, ensuring that the sum of all weights is 1. For the first The best value The corresponding normalized weights reflect the The proportion of the value in the integrated estimation.

[0074] This embodiment prioritizes the best practices. Based on the overall performance score, integration weights are allocated in a normalized manner, so that the better the performance, the better. The greater the contribution of the value in the subsequent weighted integration, the more differentiated the estimation results of different time scales are achieved, avoiding the bias that may be introduced by simple averaging, and further improving the accuracy and stability of the final state transition matrix estimation.

[0075] Based on the above embodiments, as one implementation of the present invention, the state transition matrices are weighted and averaged according to the normalized weights to obtain the final state transition matrix estimation result, which may include the following steps: The time-decoupled re-estimation step based on modulus segmentation of the full observation data is calculated to obtain The estimated transition matrix results are used to calculate the normalized weights according to the ensemble weight normalization allocation step based on cross-validation performance. By performing a weighted average and fusing the estimation results from different time scales, the final state transition matrix estimate of the adaptive time-decoupled ensemble estimator is obtained: ; in, For the optimal participation in weighted integration Total number of values. For the first The best The normalized weights corresponding to the values. To use complete observation data and the first The best The transition matrix obtained by value estimation. The final state transition matrix estimation result output by the adaptive time decoupling ensemble estimator is the core basis for attackers to infer the patroller's behavior patterns and identify patrol blind spots.

[0076] Through the weighted integration described above, estimation results from different time scales are effectively fused, with higher scoring weights being used to integrate the results. The estimated matrix corresponding to the value carries greater weight in the final result, thus reducing the impact of randomness in the selection of a single parameter and making the final estimation result more stable and reliable. Based on this final estimation matrix, attackers can accurately infer the patroller's behavioral patterns, identify the timing and location of patrol blind spots, and provide decision support for selecting the optimal attack window.

[0077] Figure 4 The results show the comparison of estimation errors between the Transition Frequency Maximum Likelihood Estimator (TFMLE) and the Adaptive Time Decoupling Integration Estimator (ATDE) under different observation scales. The horizontal axis represents the number of state transitions observed by the attacker, with a larger value indicating more sufficient observation data accumulated by the attacker. The vertical axis represents the error between the state transition matrix estimated by the attacker and the actual state transition matrix of the patroller, with a smaller value indicating higher estimation accuracy, i.e., more accurate estimation of the state transition matrix of the unmanned system patrol strategy by the attacker.

[0078] Overall, both curves show a monotonically decreasing trend with the increase of the number of observations. That is, the more sufficient the observation data, the smaller the estimation error, and the closer the state transition matrix estimated by the attacker is to the true state transition matrix of the patroller. The experimental results are consistent with the theoretical analysis.

[0079] Comparing the two algorithms, the blue curve represents the error curve of the Transition Frequency Maximum Likelihood Estimator (TFMLE), while the orange curve represents the error curve of the Adaptive Time Decoupling Integration Estimator (ATDE). Under all observation scales, the orange curve is consistently lower than the blue curve, indicating that the Adaptive Time Decoupling Integration Estimator (ATDE) outperforms the Transition Frequency Maximum Likelihood Estimator (TFMLE) under various observation conditions. By using time decoupling and weighted integration mechanisms, it effectively reduces estimation errors and significantly improves the estimation accuracy of the attacker's state transition matrix for the unmanned system patrol strategy.

[0080] This invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in this invention.

[0081] In summary, the main technical effects of this invention are as follows: (1) It fills the gap in the study of the attacker's perspective in adversarial games. Existing research on patrol of unmanned systems is all from the perspective of the defender. This invention is the first to systematically study the estimation problem of the state transition matrix of patrol strategy of unmanned system from the perspective of the attacker, improves the theoretical system of adversarial games, and provides a quantifiable analytical paradigm for evaluating the anti-prediction robustness of patrol strategy.

[0082] (2) The transition frequency maximum likelihood estimator (TFMLE) and the adaptive time decoupling ensemble estimator (ATDE) are both fully data-driven estimation methods. Attackers do not need to know any prior information about the patroller's state transition matrix, nor do they need to know whether the real matrix has a special structure such as low rank. They can gradually approximate the real transition matrix by relying only on the state sequence data accumulated through continuous observation.

[0083] (3) Adaptive Time Decoupling Integrated Estimator (ATDE) introduces a time interval parameter Temporal decoupling of the observation sequence divides the temporally coupled samples into approximately independent subsequences, fundamentally eliminating the interference of inherent temporal correlation in the Markov chain observation sequence on frequency statistics. Compared with the basic transition frequency maximum likelihood estimator (TFMLE) method, it significantly improves the estimation accuracy of the state transition matrix.

[0084] (4) Adaptive Time Decoupling Integrated Estimator (ATDE) dynamically adjusts candidate estimators based on the scale of observed data. Value search range, and through K Cross-validation automatically selects the optimal solution. The algorithm combines values ​​without requiring manual parameter specification, and can adaptively adapt to observation data of different scales, demonstrating good generalization ability and practicality.

[0085] (5) Adaptive Time Decoupling Integrated Estimator (ATDE) does not depend on a single Instead of estimating the value, it combines multiple optimal values. The estimated matrix under the value is weighted and integrated according to the performance weight. Multi-scale fusion effectively reduces the influence of the randomness of single parameter selection, making the final estimation result more stable and reliable.

Claims

1. A method for estimating an attacker's patrol strategy for an unmanned system in an adversarial environment, characterized in that, include: Attackers continuously observe the patrollers' movement trajectories and record their state sequences; Based on the state sequence, the attacker uses the transition frequency maximum likelihood estimator as the basic estimation method. The transition frequency maximum likelihood estimator estimates the state transition matrix from a state sequence by counting the frequency of transitions between states, constructing a transition count matrix and normalizing it. Building upon the framework of the aforementioned maximum likelihood estimator for transition frequencies, the attacker employs an adaptive time-decoupled ensemble estimator to perform the following steps: Based on the length of the state sequence, candidate time interval parameters are adaptively generated. A set; pass K Cross-validation, for each candidate Performance evaluation of the values: for each candidate Value, sort the state sequence according to K The dataset is divided into a training set and a validation set using a folding method. On the training set, the data is then processed according to... The state transition matrix is ​​estimated by partitioning the subset and calling the transition frequency maximum likelihood estimator. The negative log-likelihood score is then calculated on the validation set using this state transition matrix, and the result is obtained by averaging across all folds. The overall performance score is calculated, and a preset number of items with the highest overall performance scores are selected. value as optimal value; For each optimal The value is calculated using a normalized weight based on its overall performance score; For each optimal The value is used to divide the entire state sequence into modulo operations. Each subset is evaluated using the maximum likelihood estimator for the transition frequency, and the optimal values ​​are obtained by averaging the results. The state transition matrix corresponding to the value; The state transition matrices are weighted and averaged according to the normalized weights to obtain the final state transition matrix estimation result. Based on the final state transition matrix estimation result, the attacker infers the patroller's behavior patterns, identifies patrol blind spots, and determines the optimal attack window.

2. The method according to claim 1, characterized in that, The transition frequency maximum likelihood estimator estimates the state transition matrix from a state sequence by statistically analyzing the frequency of transitions between states, constructing a transition count matrix, and normalizing it. This includes: The state sequence is represented as follows: ,in T Let be the sequence length, and let 'a' be the number of states of the patroller. For each transition in the aforementioned state sequence, increment the corresponding element of the counting matrix by 1 to obtain an a×a counting matrix, where each element... Indicates from state to state The number of transfers; For each state Calculate the total number of transitions starting from this state. ,in, For the counting matrix, the first The sum of rows; For each state pair Estimated state transition probabilities for: ; when When this occurs, the state transition probability of that state is set to a uniform distribution. Representing state It never appears as the starting state in the aforementioned state sequence.

3. The method according to claim 1, characterized in that, The passage K Cross-validation, for each candidate τ The values ​​are used for performance evaluation, including: The state sequence is represented as a transition pair sequence. and in accordance with K The training set is divided into training and validation sets using folded cross-validation; for each fold of the training set, the current candidate... Values ​​divide it into There are subsets, and the time interval between adjacent transition pairs within each subset is... ; For the patrolmen in the The state at any given moment For the patrolmen in the The state at any given moment T The sequence length; On the training set, the transition frequency maximum likelihood estimator is called separately for each subset to estimate the sub-transition matrix. For a given The value is averaged over the subtransition matrices of all non-empty subsets to obtain the comprehensive estimation matrix on the current training set: ； in, This is the current cross-validation step number; Number the subsets. ∈{0,1,..., -1}, total A subset; The set of indices for non-empty subsets; Indicates the first The training set of the first fold Individuals clustered together, patrolmen from the state Transition to state The actual number of observations; Indicates the first The training set of the first fold Individuals clustered together, patrolmen from the state Total number of transfers from the starting point; Indicates the first The training set of the first fold On a subset, the transition frequency maximum likelihood estimator estimates the states from... Transition to state The probability value; Indicates the first On the training set, use When the value is obtained, the comprehensive estimated transition probability is obtained by averaging the estimated matrices of all non-empty subsets; On the corresponding validation set, the negative log-likelihood of the comprehensive estimation matrix is ​​used as the basis for this. The performance score at this discount: ； in, Indicates the first The total number of sample transition pairs in the validation set; This indicates traversing each actually observed state transition in the verification set; For the first Used in cross-validation The negative log-likelihood value; Regarding Value in all K The average of the performance scores is used to obtain the overall performance score. : ； in, K This is the total number of folds for cross-validation, and its value is dynamically adjusted based on the sequence length of the state sequence.

4. The method according to claim 3, characterized in that, The preset number of items with the highest overall performance scores are selected. value as optimal Values, including: From all candidates In the values, select the comprehensive performance score. The highest front indivual Values ​​constitute the optimal Value set: ； in, For the preset optimal Number of values as a candidate Any subset of values, The optimal selection Value set.

5. The method according to claim 4, characterized in that, The optimal The value is calculated with a normalized weight based on its overall performance score, including: For the optimal Each in the set of values value Calculate its normalized weights using the following formula: ； in, For the first The best The overall performance score is worthwhile. For all optimal The sum of the performance scores of the values for The corresponding normalized weights.

6. The method according to claim 5, characterized in that, The optimal The value is used to divide the entire state sequence into modulo operations. Each subset is evaluated by calling the maximum likelihood estimator of the transition frequency on each subset, and the optimal values ​​are obtained by averaging. The state transition matrix corresponding to the value includes: For the optimal Each optimal value in the set value , make the sequence length of T The complete state sequence is divided into modulo operations. Subset: ； in, Indicates time t Divide by The remainder is used to evenly distribute the complete state sequence to A subset; The first division based on the modulo operation A subset; For each subset, the transition frequency maximum likelihood estimator is called independently to estimate the state transition matrix, and then... The optimal state transition matrix is ​​obtained by averaging the state transition matrices of the subsets. The state transition matrix corresponding to the value .

7. The method according to claim 6, characterized in that, The step of weighting each state transition matrix according to the normalized weights to obtain the final state transition matrix estimation result includes: ； in, The final state transition matrix estimation result output by the adaptive time decoupling ensemble estimator is the core basis for attackers to infer the patroller's behavior patterns and identify patrol blind spots.

8. The method according to claim 1, characterized in that, The candidate time interval parameter is adaptively generated based on the length of the state sequence. The set includes: Determine the basic candidate value ∈{2,3,4,5,6,7}; When the length of the state sequence T When the value is greater than 100, add {8, 10, 12}; When the length of the state sequence T When the value is greater than 200, add {15, 18, 20}; When the length of the state sequence T If the value is greater than 500, further increase {25,30}.

9. The method according to claim 1, characterized in that, The patroller is an autonomous unmanned system that performs patrol missions, including drones, unmanned vehicles, or unmanned boats / vessels; the attacker is an adversary equipped with sensing devices and computing units, wherein the computing units are used to perform data processing steps of the estimation method, and the sensing devices are used to continuously observe the movement trajectory of the patroller.

10. An electronic device, characterized in that, The method includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 9.

Images