Toughness-enhanced comprehensive safety prevention and control technical architecture construction method for unmanned cluster system

Abstract

The technical scheme of the invention discloses a toughness-enhanced comprehensive safety prevention and control technical architecture construction method for an unmanned cluster system. The invention provides a new method for constructing a multi-party game mechanism model under physical-network cooperative attack, fragile nodes of a system and an attack evolution mechanism are obtained in advance based on deep reinforcement learning, and a new system is evaluated after an accident according to an expected performance index and a cooperative control effect. Meanwhile, constraint and interaction characteristics of the function-limited cluster system are fully considered, and a reference model is provided for construction of a subsequent security penetration collaborative prevention and control system. On the basis, the problem that system vulnerabilities are difficult to position in an unreliable network environment is solved. The constraint characteristic and the virtual-real interaction characteristic of the limited unmanned cluster system are fully utilized, efficient identification and quantitative evaluation of the vulnerability of the system can be realized, and meanwhile, a theoretical basis is provided for high-survivability security defense.

Description

Technical Field

[0001] The present invention relates to a method for constructing a resilience-enhanced integrated security prevention and control technology architecture for an unmanned aerial vehicle cluster system, and belongs to the technical field of security prevention and control of unmanned cluster systems. Background Art

[0002] With the rapid development of intelligent technology, physical devices are gradually evolving towards intelligence, unmannedness, and clustering, and the focus of control objectives has shifted from traditional physical space competition to the competition in the information space and intelligent algorithms. As a typical representative of swarm intelligence systems, unmanned cluster systems, with their all-digital communication and open architecture characteristics, lead the transformation of the behavior mode from traditional mechanized operations to a highly intelligent and information-based direction. However, from the external environment, with the upgrading of the offensive and defensive confrontation situation in the complex and changing application environment, the attack surface in the cyber space is continuously expanding towards diversification and intelligence, resulting in an increasingly significant information asymmetry between the offensive and defensive sides, which is bound to cause the security threats of unmanned cluster systems to escalate; from the internal environment, there are many problems in the existing security guarantee mechanisms of unmanned cluster systems, such as "unknown sources, unclear specifications, and incomplete systems", and a complete-chain, full-system, and full-coverage security application system has not been formed, inevitably there are serious security hazards.

[0003] In the face of the increasingly intense complex network offensive and defensive confrontation game environment, the security defense idea of unmanned clusters should not be limited to the security boundary only, but gradually transform into a complete and mature security sliding scale protection perspective. Specifically, the concept of "security left shift" is to pre-position the security protection work and fully consider security measures in the front-end links such as requirement analysis, design planning, and strategy generation, so as to effectively control the potential security risks of the system in advance; while the concept of "security right shift" is an important transformation direction of modern security defense strategies, that is, from the traditional mode relying on boundary defense to a mode that pays more attention to continuous monitoring, agile response, and dynamic adaptive defense mechanisms. Generally, security vulnerabilities are widely distributed in databases, operating systems, network devices, and wide area network applications, and are easily threatened by many external risks. At the same time, an increasing number of intelligent unmanned systems are exposed on the shared communication network without security authentication, becoming the "star" target of hacker attacks. From the external environment, malicious attackers launching network attack behaviors are the most common and direct risk sources. At the present stage, attackers have developed a variety of network attack means by destroying the security requirement characteristics of the system. Teixeira et al. classified typical network threats from the following three aspects:

[0004] (1) Whether the threat behavior requires prior knowledge of the system: Attackers can construct more complex and intelligent attack models based on the prior model knowledge of the system to maximize the attack ability.

[0005] (2) Whether the threat behavior has the ability to disclose critical information of the system: The attacker breaks the key data confidentiality protocol to casually obtain the critical information of the system, thus laying the foundation for implementing in-depth intrusion behavior.

[0006] (3) Whether the threat behavior has the ability to damage limited resources of the system: The attacker disrupts the transmission or tampers with information to damage the availability or feasibility of the data, thereby affecting the normal operation of the system.

[0007] In the process of the deep integration of cyber-physical systems, potential security vulnerabilities and threats in unmanned cluster systems may lead to risks such as changes in the topological structure of information systems, interruption of communication links, leakage of key information, and tampering of control instructions, which will inevitably cause serious non-contact damage to physical systems. In addition, the increasingly intelligent attack evolution mechanism makes it difficult for traditional information defense technologies to identify attack intentions, and unmanned cluster systems lack application-level protection capabilities due to their inherent vulnerabilities. Therefore, identifying the vulnerable points of the cluster system that are vulnerable to threats and thereby exploring potential attack paths and risk propagation directions is an important prerequisite for ensuring the stable operation of the system. The academic community has conducted a large number of studies on vulnerability and risk assessment in view of the security requirements of cluster systems. Yagan et al. revealed from the perspective of cascading failures that deliberate attacks on central nodes in large-scale complex networks will exacerbate the degree of system damage. Dong et al. constructed a multi-dimensional heterogeneous cyber-physical system model with weak dependence on connection edges and searched for the weak links of the system in scenarios such as physical edge removal and edge overload. In view of the hierarchical relationship and scheduling characteristics of the information network structure, Huang Jiahui et al. proposed an intelligent quantitative method for system vulnerabilities based on attack graphs and comprehensively analyzed the possible security hazards according to this technology. However, only exploring system vulnerable points from the perspective of vulnerability distribution or vulnerability utilization rate ignores the important feature of the high correlation and coupling between cyber and physical in unmanned cluster systems. Thus, Abianeh et al. proposed a vulnerability identification method based on deep reinforcement learning to automatically search for system vulnerable points by generating optimal attack sequences. Hendrickx et al. proved that system vulnerability analysis is an NP-hard problem, while Wang Yinan et al. established an attack cost function with energy constraints and regarded the minimum-cost line of the cost function as the weakest link of the system. Compared with the academic community, the research focus in the industrial field is whether frequently updated network security products can effectively resist various external risks. Usually, engineers will deploy security tools such as firewalls, virtual private networks, intrusion detection, and vulnerability scanning to monitor information systems, so as to prevent and respond to the above security risks. However, only after the risk acts on the system will the firewall start the defense response mechanism, lacking the ability to have an overall view of the security situation; the virtual private network cannot identify highly concealed attack behaviors and lacks the ability to prevent the entire process of the security situation; there are a large number of alarm events in the intrusion detection system, which may be filled with a large number of unreliable data packets; while the vulnerability scanner only conducts independent scans on hosts and lacks the ability to control the entire domain of the security situation. The above research results show that carrying out system vulnerability assessment and analyzing the risk propagation mechanism can provide theoretical support for building a security protection system, and the development and testing of network security products are the specific manifestation forms of building a security protection system for autonomous unmanned cluster systems.

[0008] Obviously, the security protection system of unmanned cluster system can be studied from two aspects: information security defense and functional safety control. The former focuses on designing a security defense system based on the idea of boundary protection from the perspective of computer science to ensure that the key facilities of the information system are protected from cyber attacks; the latter mainly starts from the perspective of control science, and designs a security control strategy with detectable, identifiable, fault-tolerant and intrusion-tolerant functions based on control theory to ensure that the cluster runs smoothly at a relatively reliable security level. In recent years, domestic and foreign researchers have formed a relatively complete technical system in dealing with model uncertainty, anti-interference and fault diagnosis. Although cyber attacks cannot directly act on physical objects like fault signals, they can also indirectly cause serious non-contact damage to physical systems through cross-domain communication networks. Therefore, the existing fault-tolerant technology is also suitable for the task of detecting or weakening cyber attacks to a certain extent. Since the intelligent unmanned cluster system is a new generation of intelligent systems produced by the organic combination of information and the physical world, its security has become a frontier scientific issue in the industry and academia. At present, researchers in the field of control have deeply analyzed the mechanism of typical attacks based on frequent network security incidents, and continuously explored the connection between intelligent network attack behaviors and the inherent security of the system. Obviously, intrusion tolerance and collaborative control have gradually become the core security defense technology for unmanned swarm systems to deal with malicious attacks. However, the existing intrusion tolerance strategies mainly target security threats brought by a single type of attack. When faced with complex situations such as physical constraints, malicious attacks, heterogeneous dynamics, and cooperative competition, the problem will become particularly complicated. Therefore, it is the key to promote the high survivability and stable operation of unmanned swarms to fully consider the behavior strategies of malicious attackers and the constraint areas of the physical system, establish a fusion attack evolution mechanism model, and build a security collaborative defense system with active intrusion tolerance based on advanced control methods to achieve consistent security status under complex multi-task requirements. Summary of the invention

[0009] The purpose of this invention is to address the problems faced by unmanned cluster systems in complex dynamic scenarios, such as frequent physical failures, diverse information attacks, and weak defense systems. Focusing on the three needs of intelligent self-discovery of network attack threats, self-assessment of cluster system security vulnerabilities, and robust active intrusion tolerance self-adjustment, a high-survivability security control architecture that can integrate threat detection, threat analysis, and threat offset is developed. A virtual-real integrated unmanned cluster security control demonstration platform is built to provide a theoretical basis and technical support for the autonomous and safe operation of unmanned clusters.

[0010] In order to achieve the above-mentioned purpose, the technical solution of the present invention is to disclose a method for constructing a comprehensive security control technology architecture with enhanced resilience for unmanned cluster systems, which is characterized by comprising:

[0011] Establish system mechanism model;

[0012] Introduce the reward functions of each agent in the system mechanism model by means of deep reinforcement learning strategies and multi-party game mechanisms, and enable continuous interaction between the attack model and the restricted system. With the goal of maximizing the sum of all cumulative individual reward values, perform multiple iterations based on the returned total reward and historical data in the experience replay pool;

[0013] Adopt a model-driven control method to estimate its own state, and then design an event-triggering mechanism under the condition of switched communication topologies by using the obtained estimated state;

[0014] Introduce the leader-follower framework of the signed communication topology graph, comprehensively consider heterogeneous unmanned cluster systems suffering from physical constraints and collaborative attacks, design an event-triggering mechanism according to the reachable set, and adopt a distributed active intrusion tolerance control strategy based on output regulation.

[0015] Preferably, in the system mechanism model, the interaction process between the agent and the external environment is described as a Markov multi-tuple model <S, A1, A2, …, A n , f, R1, R2, …, R n , γ>, where n is the number of agents, S is the state space set in the environment, A1, A2, …, A n is the action space of each agent, A = A1 × A2 × … × A n is the set of joint action spaces of all agents, f: S × A × S → [0, 1] is the joint state transition function, R i : S × A × S → R, i = 1, 2, …, n, is the reward function of each agent, R is the space set of the reward functions, and γ ∈ (0, 1) is the discount factor.

[0016] Preferably, in the deep reinforcement learning strategy and multi-party game mechanism, each subsystem is regarded as a player and endued with a selfish attribute, and the corresponding reward function R i , i = 1, 2, …, n, n is the number of agents, with the maximum damage of the restricted system under physical-network collaborative attacks as the training target of the attack strategy. When the system suffers damage, corresponding rewards for attack actions should be given, and the efficiency of the attack strategy should be adjusted and ensured according to the attack duration and energy consumption.

[0017] Preferably, when the uncertainty of the system itself and the external environment induces the switching of the communication topology among agents, design the following event-triggering communication protocol:

[0018]

[0019] where: σ(t) is a piecewise switching signal and satisfies σ(t): [0, ∞) → P, P = {1, …, N} is a finite set of switched communication graphs; is the event-triggered function to be designed, represents the state estimation error function of the current agent i, represents the measurement error of the current agent i, represents the state estimation value of the current agent i; represents the adjacency matrix, where i represents the current agent and j represents the neighbor agent j of the current agent i; represents the current triggering time of the current agent i, q represents the current time, q = 0, 1, 2, …; N i represents the set of all neighbor agents of the current agent i; N represents the number of agents in the unmanned cluster; represents the state estimation value of the j-th neighbor agent of the current agent i.

[0020] Preferably, a model-driven control method is adopted to estimate the state value of the agent, as shown in the following formula:

[0021]

[0022] In the formula: s represents the indication symbol of attack switching; k indicates that an event trigger has occurred at this time. If a certain parameter has no superscript k, it means that there is no event trigger; is the topological switching time when the attack is active, is the event triggering time when the attack is dormant; represents the system equation of agent j; represents the state quantity of neighbor agent j when there is attack switching and an event trigger occurs.

[0023] Preferably, a distributed elastic cooperative controller and a distributed active intrusion tolerance controller are used to ensure that the system can achieve a safe state one under different attack intensities.

[0024] Preferably, in the designed event triggering mechanism, agent i starts from the initial state y ∈ X i The reachable set after s seconds is usually described as:

[0025]

[0026] In the formula: s represents time, y represents the initial state; X i represents the set that the system state of the current agent i can reach; u i represents the control quantity of the current agent i; U i represents the control input set of the current agent i.

[0027] Preferably, in the distributed active intrusion tolerance control strategy based on output regulation:

[0028] In the leader-follower framework consistency framework, the dynamic leader for generating the reference trajectory can be described by an autonomous external system, i.e., represents the state equation of the leader agent, S represents the gain matrix, v(t) represents the state of the leader agent, for design a fully distributed observer for estimating leader-related information, represents the time when the event is triggered for the current agent i, and its form is:

[0029]

[0030] where: is a positive definite function related to ; is a time-varying adaptive gain; is the communication weight after being attacked, a ij is an element of the adjacency matrix; the weight d i ={-1, 1} is used to distinguish two different subgroups; represents the tracking error equation of agent i; represents the state of the leader agent at the time when the event is triggered for agent i; represents the state when the event is triggered for agent i.

[0031] Preferably, a data-driven distributed intrusion-tolerant security control strategy is also adopted.

[0032] Preferably, in the data-driven distributed intrusion-tolerant security control strategy:

[0033] Use a self-learning iterative strategy of system state and input information under the condition that the initial conditions are known, and solve the algebraic Riccati equation to obtain the optimal feedback controller gain;

[0034] Design an adaptive optimal security bipartite output consistency controller through the derived static feedback gain to achieve distributed active intrusion tolerance control when the system dynamic parameters are unknown, that is, when suffering from physical-network collaborative attacks, the follower agents of two subgroups with competitive relationships can respectively track the leader output trajectories with equal modulus and opposite signs.

[0035] The present invention proposes a new method for constructing a multi-party game mechanism model under physical-network collaborative attacks. Based on deep reinforcement learning, the vulnerable nodes of the system and the evolution mechanism of attacks are obtained in advance, and a new post-event evaluation system is given according to the expected performance indicators and collaborative control effects. At the same time, the present invention fully considers the constraints and interaction characteristics of the function-limited cluster system, providing a reference model for the construction of subsequent security penetration collaborative prevention and control systems. On this basis, the present invention solves the problem of difficult positioning of system vulnerabilities in unreliable network environments. By making full use of the constraint characteristics and virtual-real interaction characteristics of the limited unmanned cluster system, the present invention can achieve efficient identification and quantitative evaluation of system vulnerabilities, and at the same time provide a theoretical basis for high-survivability security defense. Finally, the present invention considers an elastic event-triggering mechanism that depends on security state information, and designs a novel elastic controller based on this to ensure the transient security of the system under low-intensity network attacks. For high-intensity network attacks, the present invention proposes an intrusion-tolerant controller with an adaptive penalty mechanism, which can automatically adjust the limited state to keep it away from the security constraint boundary.

[0036] Compared with the prior art solutions, the present invention has the following beneficial effects:

[0037] 1) The present invention proposes a new method for constructing a multi-party game mechanism model under physical-network collaborative attacks. Based on deep reinforcement learning, the vulnerable nodes of the system and the evolution mechanism of attacks are obtained in advance, and a new post-event evaluation system is given according to the expected performance indicators and collaborative control effects.

[0038] 2) The present invention makes full use of the constraint characteristics and virtual-real interaction characteristics of the limited unmanned cluster system, can achieve efficient identification and quantitative evaluation of system vulnerabilities, and at the same time provides a theoretical basis for high-survivability security defense.

[0039] 3) The present invention fully considers an elastic event-triggering mechanism that depends on security state information, and designs a novel elastic controller based on this to ensure the transient security of the system under low-intensity network attacks; for high-intensity network attacks, an intrusion-tolerant controller with an adaptive penalty mechanism is proposed, which can automatically adjust the limited state to keep it away from the security constraint boundary. Description of the Drawings

[0040] Figure 1 It is a flowchart of the comprehensive security prevention and control technology architecture for enhancing the resilience of the unmanned cluster system;

[0041] Figure 2 It is a schematic diagram of the distribution of vulnerable points of the unmanned cluster system;

[0042] Figure 3 It is a diagram of the elastic intrusion-tolerant control mechanism of the unmanned cluster system;

[0043] Figure 4 It is an architecture diagram of the unmanned cluster system for multi-party games;

[0044] Figure 5 Schematic diagram of elastic control for unmanned cluster system;

[0045] Figure 6 Schematic diagram of active intrusion tolerance control for unmanned cluster system;

[0046] Figure 7 Schematic diagram of external threat simulation;

[0047] Figure 8 Schematic diagram of unmanned cluster simulation verification environment. Detailed implementation manners

[0048] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of this application.

[0049] High-survivability security prevention and control technology has gradually become the core security defense technology for unmanned cluster systems to cope with malicious attacks. However, the existing security cooperation control strategies mainly target the security threats brought by single-type attacks. When facing complex situations such as physical constraints, malicious attacks, heterogeneous dynamics, and competitive interactions, this problem will become particularly complex, as Figure 2 shown. Therefore, as Figure 1 shown, the present invention fully considers the behavior strategies of malicious attackers and the constraint regions of physical systems, establishes a mechanism model integrating attack evolution mechanism and multi-party game, introduces advanced control means to construct a security cooperation defense system with active intrusion tolerance ability, and realizes the consistency of security states and outputs under complex multi-task requirements, which is the key to promoting the high-quality development of the field of security cooperation control of intelligent unmanned systems.

[0050] In order to relax the dependence of the existing security control strategy on the detection accuracy of abnormal states, the present invention proposes the concept of elastic intrusion tolerance control mechanism to represent the threat resistance ability of unmanned cluster systems, and ensures that the closed-loop system still has an acceptable operation level when coping with threats, as Figure 3 shown.

[0051] According to the distributed characteristics of each layer of the unmanned cluster system and the orderly characteristics of the overall decision-making process, the present invention introduces the Markov decision process in the optimal control theory. The interaction process between the agent and the external environment is described as a Markov multi-tuple model <S, A1, A2,..., A n , f, r1, r2,..., r n , γ>, where n is the number of agents, S is the state space set in the environment, A1, A2,..., A nis the action space for each agent, \(A = A_1\times A_2\times\cdots\times A\) n is the set of joint action spaces for all agents, \(f: S\times A\times S\rightarrow[0, 1]\) is the joint state transition function, \(R\) i : \(S\times A\times S\rightarrow R\), \(i = 1, 2,\cdots, n\), is the reward function for each agent, \(R\) is the space set of the reward function, \(\gamma\in(0, 1)\) is the discount factor. To accurately characterize the system model and improve the training effect of subsequent algorithms, it is necessary to ensure that the state space set \(S\) can represent the multi-physical constraint characteristics of the unmanned cluster system, the joint action space set \(A\) can reflect the multi-source and multi-element nature of physical-network collaborative attacks, and the joint state transition function \(f\) can describe the state changes of the system after being maliciously attacked.

[0052] After establishing the system mechanism model, a deep reinforcement learning strategy and a multi-party game mechanism are introduced to design the reward function of each agent. The reinforcement learning of the multi-agent system follows the stochastic game strategy, regarding each subsystem as a player and endowing it with a selfish attribute, and designing the corresponding reward function \(R\) according to the game interaction strategy space at each level i , \(i = 1, 2,\cdots, n\). Taking the maximum damage of the restricted system under physical-network collaborative attacks as the training objective of the attack strategy, corresponding attack action rewards should be given when the system suffers damage. In addition, to improve the identification speed of system vulnerabilities, the efficiency of the attack strategy is adjusted and ensured according to the attack duration and energy consumption. During the training stage of the deep reinforcement learning algorithm, considering factors such as the degree of system damage after the attack, the reward function \(R\) i is fed back to the attacker together with the current system state. To achieve the optimal attack strategy, it is necessary to continuously interact between the attack model and the restricted system, aiming to maximize the sum of all cumulative individual reward values, and perform multiple iterations according to the returned total reward and historical data in the experience replay pool.

[0053] Due to factors such as link failures, external interferences, and malicious attack intrusions, the unmanned cluster system will exhibit switching topology phenomena in an unreliable network environment. Considering that it is difficult for each agent to obtain its true state, the present invention first uses a model-driven control method to estimate its own state, and then designs an event-triggering mechanism under the switched communication topology using the obtained estimated state. In addition, malicious attackers will launch collaborative attack behaviors of different intensities according to actual task requirements, aiming to damage the controlled object to the greatest extent. As Figure 5 shown, the present invention separately designs a distributed elastic collaborative controller and a distributed active intrusion tolerance controller to ensure that the system can achieve consistent safe states under different attack intensities, specifically including the following steps:

[0054] 1) Design of the event-triggering mechanism for malicious attacks under switched topologies. Since the control signal of agent \(i\) needs to continuously communicate with its neighboring agent \(j\) and also continuously calculate the state \(x\)i (t) and the state x of its neighbors j (t). Therefore, designing a reasonable event-triggered control strategy can effectively avoid continuous communication between agents, thereby reducing the controller update frequency to improve network resource utilization. Considering that malicious attackers can continuously attack physical devices or communication networks to interrupt or paralyze the information interaction between agents. Therefore, when the uncertainty of the system itself and the external environment induces a switch in the communication topology between agents, the following event-triggered communication protocol is designed:

[0055]

[0056] In Equation (1): σ(t) is a piecewise switching signal and satisfies σ(t): [0, ∞) → P, where P = {1, …, N} is a finite set of switching communication graphs; is the event-triggered function to be designed, represents the state estimation error function of the current agent i, represents the measurement error of the current agent i, represents the state estimate value of the current agent i; represents the adjacency matrix, where i represents the current agent and j represents the neighbor agent j of the current agent i; represents the current trigger time of the current agent i, q represents the current time, q = 0, 1, 2, …; N i represents the set of all neighbor agents of the current agent i; N represents the number of agents in the unmanned cluster; represents the state estimate value of the j-th neighbor agent of the current agent i.

[0057] Furthermore, a model-driven control method is adopted to estimate the state value of the agent:

[0058]

[0059] In Equation (2): s represents the indication symbol of attack switching; k represents that an event trigger has occurred at this time. If a certain parameter has no superscript k, it means that no event trigger has occurred; is the topology switching time when the attack is active, is the event trigger time when the attack is dormant; represents the system equation of agent j; represents the state quantity of neighbor agent j when there is an attack switch and an event trigger occurs.

[0060] Since the state update of neighbor agent j only occurs at discrete times, and the estimated value at this time is equal to its actual state value, continuous communication between agents can be avoided, thereby making the control input signal i i(t) can be constructed as a piecewise continuous function. In the embodiments of the present invention, the control input signal is in the proportional form of PID control, which will not be elaborated here.

[0061] 2) Distributed resilient cooperative control under low-intensity attacks. Under low-intensity attacks, malicious attackers are unable to directly paralyze the system due to their limited energy. However, low-intensity malicious attacks will still lead to a decrease in the system control performance, and further cause phenomena such as system chattering and drift. Since the attack behavior divides the entire time series into a secure communication interval and an attack-active interval, a distributed resilient cooperative controller is designed using combined measurement technology in the secure communication interval, which can effectively reduce the communication load between agents and the update frequency of the controller. At the same time, the state changes of each agent in the attack-active interval are predicted using the information in the communication interval. On this basis, a novel distributed adaptive cooperative controller design method is proposed based on the state boundary layer technology, aiming to convert a discontinuous controller into a continuous control strategy through continuous approximation. Compared with fixed event-triggering conditions, the event-triggering conditions can adaptively adjust the triggering condition parameters online. The designed distributed adaptive event-triggered resilient cooperative control algorithm can not only completely avoid continuous communication between agents but also completely avoid the occurrence of Zeno behavior. In addition, based on the co-design of non-convex triggering parameters and controllers, a sequential convex optimization algorithm is introduced to obtain the optimal / sub-optimal control gains, ensuring that the impact of cooperative attacks on the system control performance is minimized.

[0062] 3) Distributed active intrusion tolerance control under high-intensity attacks. Under high-intensity attacks, the states of each agent may approach or break through the preset safety state constraint boundaries. Since agent collision avoidance and system control performance are deeply coupled, a potential energy-based control driving gain is introduced to perform path integration on the obstacle function, which can ensure that when agents approach each other under high-intensity attacks, the cooperative tracking error can be adaptively reduced to achieve collision avoidance. On this basis, considering the constraints of the system closed-loop control performance, an intelligent command adjustment method based on event triggering is proposed. Furthermore, a distributed active intrusion tolerance control strategy is designed in combination with an adaptive method, which dynamically determines whether each agent continues to execute tasks according to the current cooperative tracking error, aiming to achieve global state consensus convergence and safe collision avoidance. In addition, when the control direction is unknown, the Nussbaum function gain method is introduced to enable an adaptive penalty mechanism to be embedded in the intrusion tolerance control protocol. When the control signal approaches the safety constraint boundary under function-limited conditions, the system can self-regulate to stay away from the dangerous area to the greatest extent and achieve consistent safety states.

[0063] On the other hand, in traditional consensus strategies, a distributed control protocol is usually designed using non - negative weights among local interacting agents. However, in a cooperation - competition network, the non - cooperative or adversarial interactions between an agent and its neighboring agents make it difficult for cooperative control to work. Based on this, the present invention introduces a leader - follower framework of symbolic communication topology, comprehensively considers heterogeneous unmanned cluster systems subject to physical constraints and collaborative attacks, designs an event - triggered mechanism according to the reachable set, and then proposes a distributed active intrusion - tolerant control strategy based on output regulation and data - driven, ensuring that the system can achieve secure bipartite output consensus when the model parameters are known / unknown. The specific steps are as follows:

[0064] 1) Design of an event - triggered mechanism based on the reachable set. Under symbolic communication topology, both the physical constraints of the system itself and external malicious attacks can change the states of two opposing subgroups of agents. The reachable set can describe all possible bounded closed sets of states in the case of zero initial state of the system and bounded peak input. For this reason, the present invention introduces the concept of the reachable set to provide ideas for the design of distributed controllers, which can ensure the quantitative analysis of the system's performance at the stable domain level. Agent i starts from an initial state y ∈ X i The reachable set after s seconds can usually be described as:

[0065]

[0066] In formula (3): s represents time, y represents the initial state; X i represents the set that the current system state of agent i can reach; u i represents the control quantity of the current agent i; U i represents the control input set of the current agent i.

[0067] Therefore, if each agent can receive information such as the dynamic equations and control spaces of neighbors at each moment under constraints and attacks, and then obtain the reachable set containing the true states of neighbors, the occurrence of Zeno behavior can be completely avoided. When the reachable set of the aggregated system is inside the desired stable domain, the controller is not triggered; conversely, when it escapes from the desired stable domain, the controller is triggered.

[0068] 2) Distributed active intrusion - tolerant control based on output regulation. To achieve secure cooperative control of heterogeneous dynamic agents, the present invention introduces the idea of cooperative output regulation and designs a class of secure consensus control algorithms based on a fully distributed observer. In the leader - follower consensus framework, the dynamic leader for generating the reference trajectory can be described by an autonomous external system, that is The state equation of the leader agent is shown, \(S\) represents the gain matrix, and \(v(t)\) represents the state of the leader agent. Since the leader communicates directly with only some of the followers, the existence of malicious attacks may prevent the leader's information from being obtained normally. Therefore, for design a fully distributed observer for estimating leader-related information, represents the time when the event is triggered for the current agent \(i\), and its form is:

[0069]

[0070] In Equation (4): is a positive definite function related to ; is a time-varying adaptive gain; is the communication weight after being attacked, \(a\) ij is an element of the adjacency matrix; the weight \(d\) i =\(\{-1, 1\}\) is used to distinguish two different subgroups; represents the tracking error equation of agent \(i\); represents the state of the leader agent at the time when the event is triggered for agent \(i\); represents the state when the event is triggered for agent \(i\).

[0071] Therefore, the observer designed based on the adaptive method can be independent of the global topology matrix information, and it can ensure that each follower can obtain the leader system matrix and its state estimate value when under attack. To achieve bipartite consensus under signed topology, a positive definite diagonal matrix \(D = diag\{d\) i \} related to the grouping situation is introduced based on the structural balance condition, and the bipartite consensus problem is transformed into a traditional consensus problem for solution by using the orthogonal transformation method.

[0072] 3) Data-driven distributed intrusion-tolerant security control. In practical applications, it is often inefficient, costly, or even difficult to identify an accurate system model. When the system matrix parameters of the followers in different subgroups are unknown, the feedback gain matrix of the secure cooperative controller cannot be directly obtained by solving the algebraic Riccati equation that depends on the system matrix information. Therefore, the present invention proposes a data-driven adaptive dynamic programming method, which uses a self-learning iterative strategy of system state and input information under known initial conditions, and obtains the optimal feedback controller gain by solving the algebraic Riccati equation. Then, an adaptive optimal secure bipartite output consensus controller is designed through the derived static feedback gain to achieve distributed active intrusion tolerance control under unknown system dynamics parameters, that is, when under a physical-network collaborative attack, the follower agents in two subgroups with competitive relationships can respectively track the output trajectories of the leaders (external systems) with equal modulus and opposite signs.

[0073] Finally, to meet the requirements of minimizing the iteration period and cost in the ground semi-physical test verification, the present invention takes the collaborative operation of UAV clusters as a typical task, designs a simulation platform to support the unmanned cluster tasks, forms a virtual-real evolution system for the collaborative operation of unmanned systems, and realizes an efficient and realistic modeling of the collaborative operation scenario of unmanned clusters in the form of virtual-real evolution, such as Figure 7 shown. It provides technical support for the rapid migration of future unmanned cluster simulations to actual platforms. The research work is mainly carried out from two aspects, including the design of the unmanned cluster task demonstration simulation platform and the simulation scene approximation method combining virtual and real. The design of the simulation platform is the basis to support the rapid verification of the security intrusion tolerance algorithm for unmanned systems. Through the simulation scene approximation method combining virtual and real, various physical models in the virtual scene are corrected to make the simulation scene environment closer to the real environment.

[0074] Such as Figure 8 shown, the design of the entire unmanned cluster task simulation system adopts the idea of combining software simulation with semi-physical simulation. The software simulation adopts the Simulink+UE4+ROS scheme, and realizes the three-dimensional display and verification of the position and attitude, three-dimensional obstacle avoidance, target recognition, and cluster collaboration of UAVs in the three-dimensional simulation scene through the controller design combined with kinematic simulation, visual perception simulation, cluster strategy simulation models, and three-dimensional rendering engine. The semi-physical simulation adopts a physical platform based on PX4 / Pixhawk / NVIDIA Xavier NX, and realizes the semi-physical simulation verification by combining functions such as the simulation emulator of the simulation platform, three-dimensional motion and trajectory display, and cluster collaboration planning.

Claims

1. A method for constructing a resilience-enhanced integrated security prevention and control technology architecture for an unmanned cluster system, characterized in that, Including: Establish a system mechanism model; Introduce a deep reinforcement learning strategy and a multi-party game mechanism to design the reward functions of each agent in the system mechanism model, and enable continuous interaction between the attack model and the restricted system. With the goal of maximizing the sum of all cumulative individual reward values, perform multiple iterations based on the returned total reward and historical data in the experience replay pool; Adopt a model-driven control method to estimate its own state, and then use the obtained estimated state to design an event-triggered mechanism under the condition of switched communication topology; Introduce the leader-follower framework of the symbolic communication topology graph, comprehensively consider heterogeneous unmanned cluster systems suffering from physical constraints and coordinated attacks, design an event-triggered mechanism according to the reachable set, and adopt a distributed active intrusion tolerance control strategy based on output regulation; 2. The method for constructing a resilience-enhanced integrated security prevention and control technology architecture for an unmanned cluster system according to claim 1, wherein In the system mechanism model, the interaction process between the agent and the external environment is described as a Markov multi-tuple model <S, A1, A2, …, A n , f, R1, R2, …, R n , γ>, where n is the number of agents, S is the state space set in the environment, A1, A2, …, A n is the action space of each agent, A = A1 × A2 × … × A n is the set of joint action spaces of all agents, f: S × A × S → [0, 1] is the joint state transition function, R i : S × A × S → R, i = 1, 2, …, n, is the reward function of each agent, R is the space set of the reward function, and γ ∈ (0, 1) is the discount factor.

3. The method for constructing a resilience-enhanced integrated security prevention and control technology architecture for an unmanned cluster system according to claim 1, characterized in that, In the deep reinforcement learning strategy and multi-party game mechanism, each subsystem is regarded as a player and endued with a selfish attribute, and a corresponding reward function R is designed according to the game interaction strategy space at each level i , where i = 1, 2, …, n and n is the number of agents. The maximum damage of the constrained system under physical-network collaborative attack is taken as the training objective of the attack strategy. When the system suffers damage, corresponding rewards for attack actions should be given, and the efficiency of the attack strategy should be adjusted and ensured according to the attack duration and energy consumption 4. A method for constructing a resilience-enhanced comprehensive security prevention and control technology architecture for an unmanned cluster system, characterized in that, When the communication topology between agents is switched due to the uncertainty of the system itself and the external environment, design the following event-triggered communication protocol: where: $\sigma(t)$ is a piecewise switching signal and satisfies $\sigma(t):[0,\infty)\to P$, $P = \{1,\ldots,N\}$ is a finite set of switching communication graphs; is the event-triggering function to be designed, represents the state estimation error function of the current agent $i$, represents the measurement error of the current agent $i$, represents the state estimate of the current agent $i$; represents the adjacency matrix, where $i$ represents the current agent and $j$ represents the neighbor agent $j$ of the current agent $i$; represents the current triggering time of the current agent $i$, $q$ represents the current time, $q = 0,1,2,\ldots$; $N$ i represents the set of all neighbor agents of the current agent $i$; $N$ represents the number of agents in the unmanned cluster; represents the state estimate of the $j$-th neighbor agent of the current agent $i$.

5. A method for constructing a resilience-enhanced integrated security prevention and control technology architecture for an unmanned cluster system, characterized in that, Adopt a model-driven control method to estimate the state value of the agent, as shown in the following formula: Where: s represents the indication symbol for attack switching; k indicates that an event has been triggered at this time. If a certain parameter has no superscript k, it means that no event has been triggered; is the topology switching moment when the attack is active, is the event triggering moment when the attack is dormant; represents the system equation of agent j; represents the state quantity of neighbor agent j when there is attack switching and an event is triggered.

6. The method for constructing a resilience-enhanced integrated security prevention and control technology architecture for an unmanned cluster system according to claim 1, characterized in that Use a distributed elastic cooperation controller and a distributed active intrusion tolerance controller to ensure that the system can achieve a safe state 1 under different attack intensities.

7. A method for constructing a resilience-enhanced integrated security prevention and control technology architecture for an unmanned cluster system, characterized in that, In the designed event-triggering mechanism, the agent i starts from the initial state y ∈ X i The reachable set after s seconds is usually described as: where: s represents time, and y represents the initial state; X i represents the set that the current system state of agent i can reach; u i represents the control quantity of the current agent i; U i represents the control input set of the current agent i.

8. A method for constructing a resilience-enhanced comprehensive security prevention and control technology architecture for an unmanned cluster system, characterized in that In the distributed active intrusion tolerance control strategy based on output regulation: In the leader-follower framework consistency framework, the dynamic leader for generating the reference trajectory can be described by an autonomous external system, that is represents the state equation of the leader agent, S represents the gain matrix, v(t) represents the state of the leader agent, for design a fully distributed observer for estimating leader-related information represents the moment when the event of the current agent i is triggered, and its form is: where: is a positive definite function related to ; is a time-varying adaptive gain; is the communication weight after being attacked, and a ij is an element of the adjacency matrix; the weight d i = {-1, 1} is used to distinguish two different subgroups; represents the tracking error equation of agent i; represents the state of the leader agent when an event is triggered in agent i; represents the state of agent i when an event is triggered.

9. The method for constructing a resilience-enhanced comprehensive security prevention and control technology architecture for an unmanned cluster system according to claim 1, wherein Also adopt a data-driven distributed intrusion tolerance and security control strategy.

10. The method for constructing a resilience-enhanced comprehensive security prevention and control technology architecture for an unmanned cluster system according to claim 9, characterized in that, In the data-driven distributed intrusion tolerance and security control strategy: Under the condition that the initial conditions are known, use the self-learning iteration strategy of the system state and input information, and solve the algebraic Riccati equation to obtain the optimal feedback controller gain; Design an adaptive optimal security bipartite output consensus controller through the derived static feedback gain to achieve distributed active intrusion tolerance control when the system dynamic parameters are unknown, that is, when suffering from physical-network coordinated attacks, the follower agents of two subgroups with competitive relationships can respectively track the leader output trajectories with equal modulus and opposite magnitudes.

Images