Wind-solar-fire hydrogen storage power grid mimicry defense method based on reinforcement learning

Abstract

The invention discloses a wind, light and fire hydrogen storage power grid mimicry defense method based on reinforcement learning, and relates to the technical field of power system network security protection. The method comprises the steps of performing real-time monitoring on the wind-light-fire hydrogen storage power grid, identifying new energy specific threats and multi-energy coordination abnormity, and generating threat situation information of the wind-light-fire hydrogen storage power grid; the power grid operation state, the threat situation, the mimicry defense state and the historical performance of the wind, light, fire and hydrogen storage power grid serve as a state space, and the action space of the wind, light, fire and hydrogen storage power grid is predicted through a mimicry defense decision model; the mimicry defense decision model adopts a multi-agent reinforcement learning framework to decompose a complex action space into a plurality of agent collaborative decisions; voting is carried out on the action space according to the logic isomerism and space-time isomerism executor pool, and a target action space is determined; and mimicry defense of the wind-light-fire hydrogen storage power grid is realized through the target action space. According to the method, the defense capability of the novel power system for complex security threats is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power system network security protection, and in particular relates to a wind-solar-thermal-storage-hydrogen power grid mimicry defense method based on reinforcement learning. BACKGROUND

[0002] With the promotion of global energy transformation, the wind-solar-thermal-storage-hydrogen integrated power grid, as an important development direction of new power systems, is becoming a key technical path to support large-scale access of renewable energy. The wind-solar-thermal-storage-hydrogen integrated power grid integrates the randomness and intermittency of wind power generation, the volatility and weather dependence of photovoltaic power generation, the adjustability and inertia support of thermal power generation, the bidirectional regulation and fast response of energy storage systems, and the electrolytic load and energy conversion of hydrogen production systems, forming the significant characteristics of diversified energy structure, complex operation characteristics, and deep coupling relationship.

[0003] During the operation of the wind-solar-thermal-storage-hydrogen power grid, the control systems, communication networks, and data platforms of various energy devices face increasingly complex network security threats. The power prediction system of wind power and photovoltaic power generation may be subjected to false weather data injection attacks, affecting the accuracy of power generation plans; the charge and discharge control of energy storage systems may be maliciously manipulated, causing abnormal regulation of power grid frequency and voltage; the abnormal change of electrolytic power of the hydrogen production system as a controllable load may mask other attack behaviors or exacerbate system instability. The coupled operation of multiple energies makes it possible for local attacks to evolve into a whole-network security event through energy flow and information flow, posing a serious threat to the safe and stable operation of the power system.

[0004] Mimicry defense, as an active defense technology based on the principles of "dynamic, heterogeneous, and redundant", improves the analysis difficulty of attackers by building a constantly changing attack surface, providing a new technical approach to solving the security protection problem of complex systems. However, traditional mimicry defense systems rely on preset rules for reconstruction decisions, lack adaptive learning ability in complex threat environments, and have poor defense capabilities. SUMMARY

[0005] Therefore, it is necessary to provide a wind-solar-thermal-storage-hydrogen power grid mimicry defense method based on reinforcement learning, which overcomes the defect of lack of adaptive learning ability of traditional mimicry defense systems and effectively improves the defense capability of new power systems facing complex security threats, providing a technical guarantee for the safe and stable operation of new power systems.

[0006] The present application adopts the following technical solutions: The present application provides a wind-solar-thermal-storage-hydrogen power grid mimicry defense method based on reinforcement learning, comprising: Real-time monitoring of the wind-solar-fire-hydrogen power grid, identifying new energy-specific threats and multi-energy coordination abnormalities, generating threat situation information of the wind-solar-fire-hydrogen power grid; Taking the power grid operation state, threat situation, mimicry defense state and historical performance of the wind-solar-fire-hydrogen power grid as the state space, the action space of the wind-solar-fire-hydrogen power grid is predicted through a mimicry defense decision model; the mimicry defense decision model adopts a multi-agent reinforcement learning framework to decompose the complex action space into multiple agent collaborative decision-making; According to the logical heterogeneity and spatiotemporal heterogeneity of the executor pool, the target action space is determined by voting; Through the target action space, mimicry defense of the wind-solar-fire-hydrogen power grid is realized.

[0007] Optionally, real-time monitoring of the wind-solar-fire-hydrogen power grid, identifying new energy-specific threats and multi-energy coordination abnormalities, generating threat situation information of the wind-solar-fire-hydrogen power grid, including: Abnormal identification of each device of the wind-solar-fire-hydrogen power grid; According to the abnormal identification result, the threat situation information of the wind-solar-fire-hydrogen power grid is determined.

[0008] Optionally, the wind-solar-fire-hydrogen power grid includes a wind power system, a photovoltaic system, an energy storage system and a hydrogen production system; abnormal identification of each device of the wind-solar-fire-hydrogen power grid includes: Through the wind speed-power relationship model and the irradiance-power relationship model, an abnormal feature space is constructed, and the high-dimensional statistical characteristics and time sequence dynamic evolution law are used to identify and classify the power output abnormalities of the wind power system and the photovoltaic system; Through the pre-constructed SOC state evolution trajectory model and the charge-discharge behavior mode feature library, through multi-scale state monitoring and behavior consistency verification, malicious regulation attacks and unauthorized operations on the energy storage system are identified; Through dynamic evaluation of electrolytic efficiency and analysis of abnormal propagation of process parameters, combined with gas quality characterization and safety threshold constraints, the safety threat of the hydrogen production system is identified; Through power balance dynamic evaluation, frequency response coordination verification and voltage stability analysis, abnormal patterns of coordinated operation between the wind-solar-fire-hydrogen system are identified.

[0009] Optionally, the threat situation information of the wind-solar-fire-hydrogen power grid includes threat level, attack type, impact range and duration; The power grid operation state includes the power generation state of various devices, system frequency and voltage parameters, and device health status; The mimicry defense state includes the current executor configuration, reconstruction history, voting weight distribution and system load; The historical performance indicators include protection success rate, false alarm rate, system availability and response time.

[0010] Optionally, multiple intelligences The volume reinforcement learning framework includes: Reconstructing the intelligent agent : Responsible for making mimicry reconstruction strategy decisions, its action space is ,in, Indicates the refactoring type. Indicates the timing of the restructuring. Indicates the scope of the reconstruction. Indicates the reconstruction intensity; Execution agent : Responsible for managing heterogeneous execution entities, its action space is ,in, Indicates the first i The activation state of each execution entity Indicates the weighting coefficient. Indicates the system redundancy level. Indicates the load balancing strategy; Voting Intelligent Agent : Responsible for optimizing voting strategies, its action space is ,in, Indicate the voting rules, Indicates the exception handling strategy. Indicates the level of fault tolerance mechanism. Indicates the choice of arbitration algorithm; Resource agent: responsible for coordinating and allocating system resources; its action space is... ,in, Represents the computational resource allocation vector. Represents a communication resource allocation vector. Represents a storage resource allocation vector. This represents the network resource allocation vector.

[0011] Optionally, the multi-agent reinforcement learning framework employs a hierarchical hybrid reward function structure: in, Represents intelligent agents i Comprehensive rewards, , , , and These represent the local reward evaluations of the intelligent agent in the dimensions of security, reliability, efficiency, adaptability, and cost, respectively. For globally shared reward signals; , , , , and For the intelligent agent i The weight coefficient of the agent; The specific calculation method of each dimension reward is as follows: Security reward For comprehensive evaluation of threat defense effect, the calculation formula is: Among them, Indicates the attack success rate, which is calculated by the ratio of the number of successful intrusions to the total number of attacks in the time window; Indicates the threat detection accuracy; Indicates the false positive rate; , , The weighted coefficient; Reliability reward For evaluating system stability, the calculation formula is: Among them, Indicates the system available time, Indicates the total running time; Indicates the number of failures; Indicates the fault recovery time; And The penalty coefficient; Efficiency reward For evaluating decision response performance, the calculation formula is: Among them, Indicates the standard response time reference value, Indicates the actual response time; Indicates the resource utilization rate; The resource consumption penalty coefficient; Adaptability reward For evaluating the ability of strategy adjustment, the calculation formula is: Among them, Indicates the performance improvement amplitude, Indicates the threat intensity change; Indicates the strategy adjustment time; The time sensitivity coefficient; Cost reward For evaluating defense cost, the calculation formula is: Among them, Indicates the calculation cost, which is calculated by CPU occupancy and power consumption; denotes the communication cost, calculated by network bandwidth occupation; denotes the reconstruction cost, including the overhead of execution body switching and configuration updating; global shared reward for evaluating the overall protection effect of the system, and the calculation formula is: wherein, denotes the overall loss of the system, calculated by the comprehensive calculation of the grid frequency deviation, voltage out-of-limit and load loss; denotes the coordination effect score of the agent; denotes the number of decision conflicts; , , is a weight coefficient.

[0012] Optionally, a logical heterogeneous and spatio-temporal heterogeneous execution body pool is constructed, comprising: by deploying multiple logical heterogeneous execution bodies which are functionally equivalent but have different implementation mechanisms on a unified hardware platform; the multiple logical heterogeneous execution bodies include different control algorithms, different data processing procedures and different decision logics; by time sequence randomization, execution order disturbance and spatial deployment differentiation, multiple spatio-temporal heterogeneous execution bodies are constructed.

[0013] The application provides a wind-solar-thermal-storage-hydrogen power grid mimic defense system based on reinforcement learning, comprising: a threat perception and environment monitoring module, which performs real-time monitoring on the wind-solar-thermal-storage-hydrogen power grid, identifies new energy specific threats and multi-energy coordination abnormalities, and generates threat situation information of the wind-solar-thermal-storage-hydrogen power grid; a prediction module, which is used for taking the grid operation state, threat situation, mimic defense state and historical performance of the wind-solar-thermal-storage-hydrogen power grid as a state space, and predicting the action space of the wind-solar-thermal-storage-hydrogen power grid through a mimic defense decision model; the mimic defense decision model adopts a multi-agent reinforcement learning framework, and decomposes the complex action space into multiple agent cooperative decisions; a voting module, which is used for voting the action space according to the logical heterogeneous and spatio-temporal heterogeneous execution body pool, and determining a target action space; a defense module, which is used for realizing mimic defense of the wind-solar-thermal-storage-hydrogen power grid through the target action space.

[0014] The application provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program is executed by a processor to realize the above-mentioned wind-solar-thermal-storage-hydrogen power grid mimic defense method based on reinforcement learning.

[0015] The application provides a computer device, including a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor implements the above-mentioned wind-solar-thermal hydrogen storage power grid mimic defense method based on reinforcement learning when executing the program.

[0016] The above-mentioned at least one technical scheme adopted by the application can achieve the following beneficial effects: In the application, firstly, the accurate identification of new energy power abnormality, energy storage system malicious regulation, hydrogen production system safety threat and multi-energy coordination abnormality is performed; then, a mimic defense decision model is constructed based on a multi-agent reinforcement learning framework, a complex decision problem is decomposed into a collaborative decision problem of multiple specialized agents, and an action space is obtained; finally, the action space is voted according to a logic heterogeneous and space-time heterogeneous executor pool, result fusion is performed, adaptive reconstruction and intelligent protection of the system are realized, the method overcomes the defect that the traditional mimic defense system lacks adaptive learning ability, effectively improves the defense ability of the new power system in the face of complex security threats, and provides a technical guarantee for the safe and stable operation of the new power system. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are included to provide a further understanding of the application, constitute a part of the application and serve to explain the application together with the specification. The illustrative embodiments of the application and their description serve to explain the application without limiting the application. In the drawings:

[0018] Figure 1 A mimic defense method based on reinforcement learning for a wind-solar-thermal hydrogen storage power grid provided by the application is shown in the flowchart; Figure 2 Another mimic defense method based on reinforcement learning for a wind-solar-thermal hydrogen storage power grid provided by the application is shown in the flowchart; Figure 3 A computer device for implementing the mimic defense method based on reinforcement learning for a wind-solar-thermal hydrogen storage power grid provided by the application is shown in the schematic diagram. DETAILED DESCRIPTION

[0019] To make the purpose, technical scheme and advantages of the application clearer, the technical scheme of the application will be described in detail below with reference to the specific embodiments of the application and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0020] Traditional power grid security protection technology is mainly designed for centralized power generation systems dominated by thermal power, and has significant shortcomings in dealing with diversified threats of wind-solar-thermal-storage-hydrogen power grid. First, traditional security protection strategies mostly use static boundary protection mechanisms, protected by passive defense means such as firewalls and intrusion detection, which are difficult to deal with dynamic attacks on new energy-specific control systems. Second, existing anomaly detection methods lack a deep understanding of the unique operating mode of wind-solar-thermal-storage-hydrogen systems, and cannot effectively identify threat behaviors based on power prediction, energy storage regulation, hydrogen production load, and other new attack vectors. In addition, traditional protection systems usually use manually configured security policies and rules, which cannot adapt to the rapid changes in the operating state of wind-solar-thermal-storage-hydrogen power grid and the dynamic evolution of the threat environment, and have problems of response lag and poor adaptability.

[0021] Pseudodefense, as an active defense technology based on the principles of "dynamic, heterogeneous, and redundant", improves the difficulty of analysis for attackers by building a constantly changing attack surface, providing a new technical approach to solve the security protection problem of complex systems. However, traditional pseudodefense systems face many challenges in power system applications: first, heterogeneous implementation is difficult, the simultaneous deployment of multiple hardware architectures and operating systems is costly and highly complex in engineering; second, real-time requirements are strict, the millisecond-level response requirement of power systems may affect system performance due to excessive heterogeneity; third, reliability requirements are extremely high, the diversification of hardware and software platforms increases the number of fault points and maintenance difficulty; fourth, authentication testing is complex, the security authentication requirements of power systems make the testing and verification of multiple platforms extremely costly. In addition, traditional pseudodefense systems mostly rely on pre-set rules for reconstruction decisions, lacking the ability to adaptively learn from complex threat environments. Reinforcement learning technology, through the interaction between agents and the environment, can find the optimal strategy in a complex state space, providing important technical support for the intelligent upgrade of pseudodefense.

[0022] In view of the key technical challenges of diversified threat identification difficulty, poor adaptability of traditional static protection strategies, and lack of detection capability for new energy-specific attack patterns in wind-solar-thermal-storage-hydrogen power grid security protection, the present invention provides a pseudodefense method for wind-solar-thermal-storage-hydrogen power grid based on reinforcement learning, which builds a unified pseudodefense architecture adapted to the characteristics of wind-solar-thermal-storage-hydrogen power grid, integrates a special new energy threat detection module, and uses reinforcement learning technology to realize intelligent optimization of protection strategies, thereby effectively improving the overall security protection capability of wind-solar-thermal-storage-hydrogen power grid.

[0023] The technical solutions provided by the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0024] Figure 1 The flowchart of the pseudodefense method for wind-solar-thermal-storage-hydrogen power grid based on reinforcement learning in the present invention, specifically comprising the following steps: S101, real-time monitoring of the wind-solar-thermal-hydrogen power grid, identifying new energy specific threats and multi-energy coordination abnormalities, generating threat situation information of the wind-solar-thermal-hydrogen power grid.

[0025] The generation of threat situation information is through real-time monitoring and analysis of the running state of the power grid system, and multi-dimensional evaluation of external threats, to identify and predict the security threats faced by the power grid. Specifically, the threat situation information of the wind-solar-thermal-hydrogen power grid includes threat level, attack type, impact range and duration, and the specific way of generating these information includes:

[0026] The threat level is evaluated by multiple factors, including the severity of the attack, the degree of impact on the system, the frequency of the attack, and the current running state of the power grid. By monitoring the power grid running data, device health status and threat information of the system in real time, the system can dynamically judge the strength of the current threat and give the corresponding threat level.

[0027] The attack type is identified through in-depth analysis of real-time monitoring data of power grid devices and systems. The system uses multiple anomaly detection techniques (such as power prediction model, SOC state monitoring of energy storage system, etc.) to identify the specific type of attack, including data injection attack, malicious regulation, illegal operation, etc. By classifying the abnormal patterns of device status, the system can clearly determine the attack type.

[0028] The impact range refers to the breadth of the impact of the threat event on the power grid system. The system determines the impact range by analyzing the propagation path of the attack, the devices and regions affected by the attack, and the linkage relationship between different subsystems (such as wind power, photovoltaic, energy storage, etc.). If a threat only affects local devices or regions, the impact range is small; if the threat can affect the whole network through the propagation of energy flow or information flow, the impact range is wide.

[0029] The duration reflects the length of time the threat event affects the operation of the power grid. The system estimates the duration of the threat by recording the start and end time of the threat, as well as the changes in various indicators (such as frequency, power, etc.) of the power grid during the threat process. Long-term threats may cause the system to remain unstable, while short-term threats may have limited impact on the system.

[0030] Optionally, identifying new energy specific threats and multi-energy coordination abnormalities includes: accurate identification of new energy power abnormalities, energy storage system malicious regulation, hydrogen production system security threats and multi-energy coordination abnormalities.

[0031] Specifically, the wind-solar-fire-hydrogen power grid is monitored in real time, threats specific to new energy and multi-energy coordination abnormalities are identified, and threat situation information of the wind-solar-fire-hydrogen power grid is generated, including: identifying abnormalities of each device of the wind-solar-fire-hydrogen power grid; determining the threat situation information of the wind-solar-fire-hydrogen power grid according to the abnormal identification result.

[0032] In one embodiment, the wind-solar-fire-hydrogen power grid includes a wind power system, a photovoltaic system, a thermal power system, an energy storage system, and a hydrogen production system; the abnormalities of each device of the wind-solar-fire-hydrogen power grid are identified, including the following steps: S201, constructing an abnormal feature space through a wind speed-power relationship model and an irradiance-power relationship model, and identifying and classifying the power output abnormalities of the wind power system and the photovoltaic system by using high-dimensional statistical characteristics and time series dynamic evolution rules.

[0033] Specifically, for new energy power detection: a multi-modal perception and cross-domain reasoning framework is constructed, which integrates physical mechanisms and data-driven methods, an abnormal feature space is constructed through a wind speed-power relationship model and an irradiance-power relationship model, and high-dimensional statistical characteristics and time series dynamic evolution rules are used to realize accurate identification and classification of wind power system and photovoltaic system power output abnormalities, and determine the threat severity of the wind power system and the photovoltaic system.

[0034] In one embodiment, a comprehensive detection framework based on multi-modal perception and cross-domain reasoning is constructed, which adopts a hierarchical architecture design, including a physical model layer, a feature extraction layer, an abnormality identification layer, and a decision fusion layer, forming a complete abnormality detection closed loop. This framework can identify the security threats specific to the wind-solar-fire-hydrogen power grid, filling the gap in traditional abnormality detection technology.

[0035] The physical model layer establishes a wind speed-power relationship model and an irradiance-power relationship model based on first principles, and constructs a theoretical benchmark for new energy power generation through aerodynamics theory and photoelectric conversion mechanism. This layer also integrates environmental factors, equipment states, and electrical parameters to establish a multi-dimensional physical constraint model, providing a reliable theoretical basis for abnormality detection.

[0036] The feature extraction layer realizes the spatio-temporal feature extraction of multi-source data, including time series fluctuation features, frequency domain spectrum features, and spatial distribution features. Through multi-dimensional cross-analysis such as vibration-power correlation analysis, temperature-power coupling modeling, and rotational speed-power consistency verification, subtle abnormal signs can be captured. For the photovoltaic system, special models such as IV characteristic curve analysis, inverter efficiency monitoring, shadow shading detection, and component aging identification are constructed, forming a comprehensive feature space.

[0037] The anomaly detection layer integrates a four-pronged approach: physical modeling, statistical analysis, machine learning, and deep learning. The statistical analysis component utilizes distribution characteristics and temporal patterns for initial screening; the machine learning method identifies anomalies with known patterns by training multiple classification and clustering models; and the deep learning method focuses on capturing complex, high-dimensional, nonlinear anomaly patterns. This fusion of multiple methods enables the system to handle various known and unknown anomaly scenarios.

[0038] The decision fusion layer employs an adaptive weighting mechanism to integrate the results of multiple detection algorithms. It dynamically adjusts the weights of each method based on historical accuracy, current environmental conditions, and system status, achieving optimal fusion of detection results. This layer also establishes a multi-level alarm mechanism, classifying and processing alarms according to their threat severity, duration, and impact scope, effectively reducing false alarm rates and improving detection reliability.

[0039] Among them, the severity of the threat to wind power systems : in, Indicates the severity of the threat posed by wind power systems; , , The wind power threat weighting coefficient satisfies ; Indicates wind power deviation; Indicates the rated power of wind power; Indicates the effective value of vibration; Indicates the vibration threshold; Indicates the gearbox temperature; This indicates the maximum permissible temperature.

[0040] Severity of threats to photovoltaic systems : in, Indicates the severity of the threat posed to photovoltaic systems; , , The photovoltaic threat weighting coefficient satisfies... ; Indicates photovoltaic power deviation; Indicates the rated power of the photovoltaic system; Indicates photovoltaic efficiency deviation; Indicates the nominal photovoltaic efficiency; Indicates the temperature of the photovoltaic module; Indicates the nominal operating temperature; This indicates the maximum permissible temperature.

[0041] S202, through the pre-constructed SOC state evolution trajectory model and the charge and discharge behavior mode feature library, through multi-scale state monitoring and behavior consistency verification, malicious regulation attacks and unauthorized operations received by the energy storage system are identified.

[0042] Specifically, for energy storage system abnormal behavior detection: based on electrochemical system dynamics and information physical system safety theory, SOC state evolution trajectory model and charge and discharge behavior mode feature library are constructed, malicious regulation attacks and unauthorized operations are identified through multi-scale state monitoring and behavior consistency verification, predictive defense of energy storage system security threats is realized, and the threat severity of energy storage system is determined.

[0043] In one embodiment, an energy storage system anomaly detection framework is constructed, which is based on electrochemical system dynamics and information physical system safety theory, and a multi-level safety monitoring system from battery monomer to system level is constructed. Through the two main lines of state monitoring and behavior analysis, the framework realizes comprehensive identification of potential threats to the energy storage system.

[0044] The state monitoring part focuses on the evolution trajectory of the battery state of charge (SOC). This method establishes a multi-parameter fusion SOC estimation model, combines multi-dimensional indicators such as internal resistance change, temperature distribution, voltage consistency and cycle life, and constructs a comprehensive evaluation system of battery health status. Through real-time monitoring and historical trend analysis of these parameters, the system can early detect battery performance abnormalities and potential safety hazards.

[0045] The behavior analysis part focuses on the timing characteristics and control logic of the charging and discharging process. This part establishes a behavior monitoring mechanism across time scales, from microsecond-level current mutations to minute-level load responses, covering the dynamic behavior characteristics of the energy storage system. Through power spectral density analysis, control instruction-response consistency verification and energy balance detection, malicious regulation attacks and unauthorized operations can be identified, effectively preventing various security threats to the energy storage system.

[0046] This embodiment also realizes multi-battery cooperative state monitoring, which identifies system-level problems that may be caused by single abnormal cells by analyzing the consistency and cooperation within the battery pack. This mechanism not only improves the sensitivity of abnormal detection, but also enhances the fault tolerance of local faults, significantly improving the overall safety of the energy storage system.

[0047] wherein the threat severity of the energy storage system : wherein, the threat severity of the energy storage system is represented; , , , is a storage energy threat weight coefficient, satisfying ; represents a state of charge deviation; represents a state of charge variation range; represents a charge-discharge efficiency; represents a current deviation; represents a rated current; represents a battery temperature; represents a maximum allowable battery temperature.

[0048] S203, by electrolytic efficiency dynamic evaluation and process parameter abnormality propagation analysis, combined with gas quality characterization and safety threshold constraint, the safety threat of the hydrogen production system is identified.

[0049] Specifically, for hydrogen production system operation anomaly detection: a multivariate collaborative monitoring framework based on the mechanism of water electrolysis process is established, through electrolytic efficiency dynamic evaluation and process parameter abnormality propagation analysis, combined with gas quality characterization and safety threshold constraint, a three-dimensional protection system throughout the whole production process is formed, realizing real-time identification of safety threats of hydrogen production system, and determining the threat severity of hydrogen production system.

[0050] In one embodiment, a hydrogen production system anomaly detection framework is constructed, which is based on the mechanism of water electrolysis process and constructs a whole-process monitoring system covering electrolysis process, gas quality and safety protection. Through multivariate collaborative monitoring and multi-level early warning mechanism, accurate identification and predictive defense of safety threats of hydrogen production system are realized.

[0051] Electrolysis process monitoring focuses on the dynamic changes of electrolysis efficiency and key parameters. This embodiment establishes a hydrogen production theoretical model based on Faraday's law, combined with electrolytic cell voltage analysis, current density distribution monitoring, membrane performance detection and electrode catalytic activity analysis, etc. in multiple dimensions, a comprehensive monitoring network of electrolysis process is constructed. Through real-time analysis and mutual verification of these parameters, the system can quickly identify process problems such as electrolysis efficiency anomaly and electrode performance decline.

[0052] The gas quality and safety monitoring part focuses on the safety and purity guarantee of hydrogen production. This part establishes a multi-level early warning system of hydrogen purity monitoring, pressure safety management, leakage detection, temperature field monitoring and equipment vibration analysis, forming a three-dimensional monitoring network from gas molecular characteristics to equipment macroscopic state. The system realizes early identification and prevention of various safety hazards by establishing safety threshold and dynamic change model of key parameters.

[0053] The embodiment also establishes a coordination analysis mechanism for hydrogen production load and power grid scheduling, which ensures that the hydrogen production system will not become a weak link in power grid safety while providing energy conversion services by monitoring the response characteristics and adjustment flexibility of hydrogen production power. This power grid-hydrogen production collaborative safety monitoring mechanism provides important safety protection for the wind-solar-thermal-storage-hydrogen integrated system.

[0054] wherein the threat severity of the hydrogen production system : wherein, indicates the threat severity of the hydrogen production system; 、 、 、 is a hydrogen production threat weight coefficient, satisfying ; indicates the electrolysis efficiency deviation; indicates the nominal electrolysis efficiency; indicates the hydrogen pressure; indicates the maximum allowable pressure; indicates the hydrogen production yield deviation; indicates the rated hydrogen production yield; indicates the impurity concentration; indicates the maximum allowable impurity concentration.

[0055] S204, through power balance dynamic evaluation, frequency response coordination verification and voltage stability analysis, abnormal modes of coordinated operation of the wind-solar-thermal-storage-hydrogen system are identified.

[0056] Specifically, for multi-energy coordination anomaly detection: a system-level anomaly identification mechanism based on energy flow-information flow double coupling is constructed, through power balance dynamic evaluation, frequency response coordination verification and voltage stability analysis, abnormal modes of coordinated operation of the wind-solar-thermal-storage-hydrogen system are identified, to prevent cascading propagation and diffusion of cross-system composite attacks.

[0057] In one embodiment, a multi-energy coordination anomaly detection framework is constructed, which is based on energy flow-information flow double coupling theory and constructs a system-level coordination anomaly identification mechanism. Through comprehensive analysis of power balance, frequency response and voltage stability, accurate identification and early warning of abnormal operation of the wind-solar-thermal-storage-hydrogen system are realized.

[0058] Power balance monitoring focuses on the energy balance state of the whole system, and establishes a multi-time scale detection model. From the instantaneous power balance to the long-term power trend, the system builds a complete power balance monitoring chain by analyzing the power imbalance, prediction error and time series characteristics. This hierarchical monitoring mechanism can distinguish between normal power fluctuations and potential abnormal disturbances, effectively preventing attacks based on power imbalance.

[0059] Frequency response coordination detection focuses on the response characteristics of the system to frequency changes, and establishes a multi-level response verification model from first-order frequency to third-order frequency. By monitoring the frequency response characteristics of wind power, energy storage and hydrogen production systems, analyzing the AGC command execution effect, and verifying the economic dispatch execution, the system can comprehensively evaluate the frequency stability and coordination of the multi-energy system, and timely identify frequency response abnormalities and dispatch execution deviations.

[0060] Voltage regulation coordination detection focuses on the voltage stability and reactive power coordination of the system, and establishes a static and dynamic voltage safety monitoring mechanism. By analyzing the voltage stability margin, post-fault voltage recovery process, and the matching of reactive power output and voltage regulation demand, it can evaluate the voltage stability risk and prevent voltage instability events from occurring.

[0061] This embodiment also establishes a cross-system fault propagation detection mechanism, which builds a cascading fault identification model by analyzing fault diffusion paths, oscillation modes and protection system coordination. This mechanism can timely discover and isolate potential cascading fault sources, prevent local abnormalities from evolving into global events, and significantly enhance the system resilience of wind-solar-thermal-storage-hydrogen power grids.

[0062] wherein the threat severity of multi-energy coordination : wherein, represents the threat severity of multi-energy coordination; , , , is the coordination threat weight coefficient, satisfying ; represents the power balance deviation; represents the total system power; represents the grid frequency deviation; represents the nominal grid frequency; represents the grid voltage deviation; represents the nominal grid voltage; represents the synchronization deviation index.

[0063] According to the threat severity of each wind power system, photovoltaic system, thermal power system, energy storage system, hydrogen production system and multi-energy coordination, the threat level of the wind-solar-thermal-storage-hydrogen power grid is determined, and the threat level calculation method is as follows: Among them, indicates the threat level of the wind-solar-thermal-storage-hydrogen power grid; indicates the threat severity of the wind-solar-thermal-storage-hydrogen power grid; indicates the threat frequency; indicates the threat influence range; indicates the threat duration; , , , is the corresponding weight coefficient, satisfying .

[0064] The threat severity of the wind-solar-thermal-storage-hydrogen power grid The calculation formula is: Among them, , , , , indicate the threat severity of the wind power system, photovoltaic system, energy storage system, hydrogen production system and multi-energy coordination respectively.

[0065] S102, the grid operation state, threat situation, mimic defense state and historical performance of the wind-solar-thermal-storage-hydrogen power grid are taken as the state space, and the action space of the wind-solar-thermal-storage-hydrogen power grid is predicted through a mimic defense decision model; the mimic defense decision model adopts a multi-agent reinforcement learning framework, and the complex action space is decomposed into multiple agent collaborative decision-making.

[0066] Among them, the grid operation state includes the power generation state of various types of equipment, system frequency voltage parameters and equipment health status; the mimic defense state includes the current executive configuration, reconstruction history, voting weight distribution and system load; the historical performance index includes the protection success rate, false alarm rate, system availability and response time.

[0067] The application adopts a multi-agent reinforcement learning framework (Multi-Agent Reinforcement Learning Framework, MARL) to solve the mimic defense decision problem, decomposes the complex defense decision space into multiple professional agent collaborative decision problems, and realizes intelligent optimization and adaptive adjustment of the defense strategy. The framework is composed of four types of professional agents, forming a complete decision ecosystem, which conforms to the dynamic heterogeneous redundancy architecture of the mimic defense principle.

[0068] Specifically, the multi-agent reinforcement learning framework decomposes the complex action space into multiple agent collaborative decision-making; the multi-agent reinforcement learning framework includes: a reconstruction agent , an execution agent , a voting agent , and a resource agent .

[0069] The reconstruction agent : responsible for mimicry reconstruction strategy decision-making, whose action space is , wherein, represents the reconstruction type, represents the reconstruction timing, represents the reconstruction range, represents the reconstruction strength.

[0070] The reconstruction agent , as the core of defense strategy decision-making, formulates mimicry reconstruction strategies based on threat situation information assessment, the reconstruction type includes logical reconstruction, space-time reconstruction, interface reconstruction, etc., the reconstruction timing includes periodic, event-triggered, hybrid-triggered, and predicted-triggered, the reconstruction range includes three levels of local, regional, and global, and the reconstruction strength includes three levels of low, medium, and high. This agent directly responds to changes in threat levels, autonomously formulates priority allocation strategies and conflict resolution mechanisms.

[0071] The execution agent : responsible for heterogeneous execution body management, whose action space is , wherein, represents the activation state of the i th execution body, represents the weight distribution coefficient, represents the system redundancy level, represents the load balancing strategy.

[0072] The execution agent is responsible for the selection and configuration management of heterogeneous execution bodies, which dynamically adjusts the execution body combination according to the reconstruction strategy to maximize the system heterogeneity.

[0073] The voting agent : responsible for voting strategy optimization, whose action space is , wherein, represents the voting rule, represents the exception handling strategy, represents the fault tolerance mechanism level, represents the arbitration algorithm selection.

[0074] Among them, the voting agent works in parallel with the heterogeneous executor, realizes the multi-mode decision function, and its action space covers voting rules such as simple majority, weighted majority, dynamic rules, abnormal handling strategies including elimination, weight reduction, correction, fault tolerance mechanism level setting, and arbitration algorithm selection. The agent analyzes the executor output in real time, dynamically adjusts the voting strategy and fault tolerance threshold.

[0075] Resource agent : responsible for the coordinated allocation of system resources, and its action space is , wherein, represents the calculation resource allocation vector, represents the communication resource allocation vector, represents the storage resource allocation vector, represents the network resource allocation vector.

[0076] Among them, the resource agent provides dynamic resource support, is responsible for the management of the allocation vector of computing, communication, storage and network resources, optimizes resource configuration in real time according to the resource demand of other agents and the system load state, and ensures the efficient execution of defense decisions.

[0077] In one embodiment, in the offline knowledge construction stage, the system first collects the historical operation data of the wind-solar-fire-storage-hydrogen power grid, including threat event records in the past three years, system response data, defense strategy execution effect, etc. Classify and label the threat events, establish a threat type library, including new energy power abnormality, malicious adjustment of energy storage, hydrogen production safety threat, multi-energy coordination abnormality, etc.; analyze the characteristic mode, occurrence frequency, influence range and duration of each type of threat. Quantitatively evaluate the execution effect of the defense strategy, and count the success rate, response time, resource consumption, etc. of different defense strategies; analyze the reasons and patterns of strategy failure. Establish a threat-defense strategy association database to record the performance of various defense strategies under specific threat types, providing basic data for strategy optimization.

[0078] Then, the digital twin model of the wind-solar-fire-storage-hydrogen power grid is established to accurately simulate the physical characteristics and operation mechanisms of each subsystem. The wind turbine system model includes aerodynamic characteristics, electrical characteristics, etc.; the photovoltaic system model includes photoelectric conversion characteristics, temperature characteristics, etc.; the energy storage system model includes electrochemical characteristics, thermal characteristics, control characteristics, etc.; the hydrogen production system model includes electrolysis process characteristics, gas transmission characteristics, safety characteristics, etc. A multi-energy system coupling model is established to simulate the energy exchange, information interaction, coordinated control, etc. between subsystems. A threat injection module is constructed, which can simulate various network attacks, device failures, coordination abnormalities, etc. in the digital twin environment, providing a rich test environment for strategy training.

[0079] The neural network parameters of the agent are continuously updated using the gradient descent method. The reconstruction agent uses the policy gradient method to adjust the policy network parameters based on the probability of action selection and the reward obtained. The execution agent uses the Actor-Critic method to simultaneously update the policy network and value network parameters. The voting agent uses the deep Q-learning method to update the network parameters based on the Q-value error. The resource agent uses a multi-objective optimization method to find a balance among multiple resource allocation objectives. An adaptive learning rate mechanism is designed to dynamically adjust the learning rate based on the learning progress. A larger learning rate is used in the early stage of learning to quickly converge, and a smaller learning rate is used in the later stage of learning to fine-tune.

[0080] Based on the correlation analysis of threat characteristics and defense strategies, a multi-dimensional decision boundary is established. The threat severity boundary is divided into three levels: low, medium, and high, each corresponding to a different combination of defense strategies. The threat impact range boundary is divided into three levels: local, regional, and global, with different resource allocation strategies for each level. The threat duration boundary is divided into three stages: short-term, medium-term, and long-term, with different defense intensities for each stage. A strategy selection decision tree is established to quickly determine the initial defense strategy based on the combination of threat characteristics. A strategy adjustment rule library is established to define the conditions under which defense strategies should be adjusted, as well as the direction and magnitude of adjustment.

[0081] In the dynamic feedback optimization phase, the system establishes a real-time interaction interface between the agent and the wind-solar-thermal-hydrogen power grid environment. It obtains system operation state, threat situation information, and defense execution effect data every second. A state perception module is designed to convert continuous environmental state into discrete state vectors for easy processing by the agent. An action execution module is designed to convert the agent's decision actions into specific defense operation instructions. A reward calculation module is designed to calculate the reward signal in real time based on the defense effect. An experience buffer is established to store state transition experiences in the recent period, including current state, selected action, obtained reward, next state, and other information, providing training samples for policy updating.

[0082] A multi-dimensional evaluation signal system is designed to comprehensively evaluate the effectiveness of defense strategies. Safety evaluation includes threat detection accuracy, defense success rate, and false alarm rate. Reliability evaluation includes system availability, fault recovery time, and continuous operation time. Efficiency evaluation includes response time, resource utilization, and energy consumption ratio. Adaptability evaluation includes new threat identification capability, strategy adjustment speed, and learning convergence speed. Cost evaluation includes computational resource consumption, communication overhead, and storage space occupation. An evaluation signal fusion mechanism is established to fuse multiple-dimensional evaluation results into a comprehensive reward signal to guide agent learning.

[0083] The neural network parameters of the agent are continuously updated using the gradient descent method. The reconstructed agent uses the policy gradient method to adjust the policy network parameters based on the probability of action selection and the reward obtained. The executing agent uses the Actor-Critic method to simultaneously update the policy network and value network parameters. The voting agent uses the deep Q-learning method to update the network parameters based on the Q-value error. The resource agent uses a multi-objective optimization method to find a balance among multiple resource allocation objectives. An adaptive learning rate mechanism is designed to dynamically adjust the learning rate based on the learning progress. A larger learning rate is used in the early stage of learning to quickly converge, and a smaller learning rate is used in the later stage of learning to fine-tune.

[0084] A strategy exploration and utilization balancing mechanism is established to continue exploring new strategy space based on known effective strategies. An ε-greedy strategy is used to control the exploration probability, maintaining a high exploration rate in the early stage of training and gradually reducing the exploration rate as training progresses. The selection frequency of different actions is balanced, with higher uncertainty actions being preferentially selected for exploration. A strategy distribution updating mechanism is established to dynamically adjust the selection probability of each strategy based on its execution effect. A strategy elimination mechanism is established to remove strategies that have performed poorly for a long time from the candidate set to make room for new strategies.

[0085] In the adaptive generalization stage, a new threat identification mechanism based on anomaly detection is established to identify potential new attacks by monitoring the deviation of threat features from known patterns. One-Class SVM is used to establish the boundary of normal threat patterns, and new attacks are marked when new threat features exceed the boundary. The Isolation Forest algorithm is used to identify outliers in the threat feature space. The autoencoder reconstruction error is used to identify new threat patterns. A threat similarity calculation mechanism is established to compare the similarity of new threats with known threats, and find the most similar known threat type as the reference for the initial response strategy.

[0086] A strategy generation mechanism is established to generate new defense strategies based on existing strategy combinations. Genetic algorithm crossover and mutation operations are used to combine the excellent parts of different strategies to generate new strategy candidates. Reinforcement learning meta-learning methods are used to learn how to quickly adapt to new threat environments. Transfer learning techniques are used to transfer knowledge learned in similar threat scenarios to the response to new threats. A strategy effectiveness rapid evaluation mechanism is established to quickly evaluate the feasibility of new strategies through small-scale experiments to avoid negative impacts of ineffective strategies on the system.

[0087] A strategy space reconstruction mechanism is established to periodically analyze and reconstruct the entire strategy space. Redundant parts in the strategy space are identified, and similar functions are merged. Blank areas in the strategy space are identified, and new strategies are generated to fill the blanks. Conflict relationships between strategies are identified, and a strategy coordination mechanism is established to avoid conflicts. Cluster analysis is used to group similar strategies, making it easier to manage and select. Dimensionality reduction techniques are used to map high-dimensional strategy spaces to low-dimensional spaces, making it easier to visualize and understand.

[0088] In the robustness guarantee system construction phase, a strategy version management system is established to manage the versions of each agent's strategy. Each strategy version is assigned a unique identifier, and information such as version creation time, update content, and performance indicators is recorded. A strategy inheritance relationship diagram is established to record the evolution relationship between strategy versions. A strategy branch management mechanism is established to support the parallel development of multiple strategy versions. A strategy rollback mechanism is designed to quickly roll back to a stable historical version when a new version performs poorly. A strategy merging mechanism is established to combine the excellent features of different branches into the main version. A strategy comparison mechanism is established to compare the performance differences between different versions. A comprehensive performance evaluation index system is established to evaluate strategy quality from multiple dimensions. Functional indicators include threat detection rate, defense success rate, false positive rate, and false negative rate. Performance indicators include response time, processing delay, throughput, and resource utilization. Reliability indicators include system availability, fault recovery time, error rate, and stability. Security indicators include the number of security vulnerabilities, security event handling effectiveness, and attack resistance. A performance benchmark test environment is established, and standardized test scenarios and test data are designed to ensure the consistency and comparability of performance evaluation. A performance monitoring dashboard is established to display the trends of various performance indicators in real time. A performance report generation mechanism is established to generate detailed performance analysis reports regularly. Further deployment of real-time monitoring systems is required to continuously track various indicators during strategy execution. An anomaly detection mechanism is established to monitor abnormal changes in performance indicators using statistical process control methods. When the indicators deviate from the normal range, an early warning is issued. A trend analysis mechanism is established to analyze the long-term trends of performance indicators and predict potential performance problems. A strategy health assessment model is designed to consider multiple performance indicators and calculate the overall health score of the strategy. A strategy lifecycle management mechanism is established to determine the retention, update, or elimination of strategies based on their health scores.

[0089] This embodiment designs a multi-stage adaptive learning architecture, realizing a complete closed loop from static knowledge acquisition to dynamic strategy optimization. Through the four stages of offline knowledge construction, dynamic feedback optimization, adaptive generalization, and robustness guarantee, a continuous evolution mechanism for system defense capabilities is formed.

[0090] (1) Offline knowledge construction phase The offline knowledge construction phase constructs the initial parameter space and decision boundary of the defense strategy through large-scale historical data analysis and high-precision digital twin environment. First, historical attack data and system operation data are collected and analyzed to establish threat models and system dynamic characteristic models, forming an initial knowledge base. Then, various attack scenarios and system states are simulated in the digital twin environment, and the defense strategy is pre-trained and evaluated through a large number of simulation experiments to identify the strengths and limitations of the strategy.

[0091] This phase uses a combination of supervised learning and reinforcement learning to guide the formation of the initial strategy based on historical experience and verify the effectiveness of the strategy in the simulation environment. This prior knowledge-based training method greatly accelerates the learning process, enabling the system to have strong defense capabilities at the initial deployment stage and shortening the cold start time for strategy optimization.

[0092] (2) Dynamic feedback optimization phase The dynamic feedback optimization phase uses real-time environment interaction and evaluation signals to continuously adjust the decision function parameters and strategy distribution to improve the system's response accuracy to actual threats. This phase uses online learning methods to record state transitions, action selection, and reward signals during actual operation as experience data and optimizes strategy network parameters based on these data.

[0093] An experience replay buffer is established to store recent interaction experiences, and a batch sampling method is used to improve sample efficiency. The strategy update uses a policy gradient-based method to adjust network parameters by calculating policy gradients and value function gradients, causing the strategy to evolve towards higher returns. In addition, a periodic evaluation mechanism is established to regularly detect strategy performance, ensuring the stability and effectiveness of the optimization process.

[0094] (3) Robustness guarantee system The robustness guarantee system establishes a multi-level strategy version control and performance evaluation framework, enabling continuous monitoring of strategy quality and safe degradation recovery in abnormal states. This system first establishes a strategy verification mechanism to verify the safety boundary and performance guarantee of the strategy using formal methods, ensuring that strategy updates do not introduce new vulnerabilities. Then, a strategy rollback mechanism is implemented to quickly recover to a stable version when performance degradation is detected, ensuring the continuous availability of the system.

[0095] A multi-strategy combination method is also used to integrate multiple strategy versions to form a strategy combination, improving the robustness and generalization ability of decision-making. In addition, an abnormal strategy detection mechanism is established to monitor abnormal changes in strategy behavior and prevent strategies from being maliciously manipulated or contaminated. This multi-level robustness guarantee design ensures that the system maintains stable defense capabilities when facing various uncertainties, significantly improving the reliability and security of the system.

[0096] In one embodiment, the multi-agent reinforcement learning framework adopts a hierarchical hybrid reward function structure: ; wherein, represents the comprehensive reward of the agent i , , , , and respectively represent the local reward evaluation of the agent in the safety, reliability, efficiency, adaptability and cost dimensions; is a global shared reward signal used to guide the overall performance improvement of the multi-agent; , , , , and are weight coefficients of the agent i , which realize the collaborative optimization of individual goals and system goals through dynamic adjustment.

[0097] The specific calculation methods of each dimension reward are as follows: Safety reward is used to comprehensively evaluate the threat defense effect, and the calculation formula is: wherein, represents the attack success rate, which is calculated by the ratio of the number of successful intrusions to the total number of attacks in a time window; represents the threat detection accuracy; represents the false positive rate; , , are weighting coefficients; Reliability reward is used to evaluate the system operation stability, and the calculation formula is: wherein, represents the system available time, represents the total running time; represents the number of failures; represents the fault recovery time; and are penalty coefficients; Efficiency reward is used to evaluate the decision response performance, and the calculation formula is: wherein, represents the standard response time reference value, represents the actual response time; represents resource utilization rate; is a resource consumption penalty coefficient; adaptive reward for evaluating the policy adjustment capability, and the calculation formula is: wherein, represents performance improvement range, represents threat intensity change; represents policy adjustment time consumption; is a time sensitivity coefficient; cost reward for evaluating the defense cost, and the calculation formula is: wherein, represents calculation cost, calculated by CPU occupancy and power consumption; represents communication cost, calculated by network bandwidth occupancy; represents reconstruction cost, including the overhead of execution body switching and configuration updating; global shared reward for evaluating the overall protection effect of the system, and the calculation formula is: wherein, represents overall system loss, calculated by power grid frequency deviation, voltage out-of-limit and load loss; represents agent cooperation effect score; represents decision conflict frequency; , , is a weight coefficient.

[0098] In the present application, a multi-agent reinforcement learning framework is adopted to construct a quasi-state defense decision model, realizing autonomous learning and continuous optimization of the protection strategy. By decomposing the complex decision space into an architecture for collaborative decision-making of multiple specialized agents, the optimal protection strategy can be learned more efficiently, and stronger adaptability and robustness can be shown when dealing with complex dynamic threat environments. The reconstruction agent is responsible for quasi-state reconstruction strategy decision-making, the execution agent is responsible for heterogeneous execution body management, the voting agent is responsible for voting strategy optimization, and the resource agent is responsible for system resource coordination and allocation. The framework adopts a hierarchical hybrid reward function structure, which maintains the consistency of the overall defense target while providing a clear optimization direction for each specialized agent, realizing the organic unification of local decision and global defense performance.

[0099] S103, voting on the action space according to the logic heterogeneous and spatio-temporal heterogeneous execution body pool to determine the target action space.

[0100] In one embodiment, for the diversification characteristics of the wind-solar-hydrogen-storage power grid, heterogeneity is established at the control algorithm and software platform level, forming a diversified pool of executors. This pool of executors forms a collection of heterogeneous executors with equivalent functions but different internal mechanisms by implementing diverse control algorithms on a unified software platform. The core concept of the pool of executors is to create a group of execution units with the same function but different structure, so that the system exhibits consistent functional characteristics externally while maintaining high structural diversity internally.

[0101] Building a logically heterogeneous and spatio-temporally heterogeneous pool of executors includes: deploying multiple logically heterogeneous executors with equivalent functions but different implementation mechanisms on a unified hardware platform; the multiple logically heterogeneous executors include different control algorithms, different data processing procedures, and different decision-making logics; and constructing multiple spatio-temporally heterogeneous executors through temporal randomization, execution order disturbance, and spatial deployment differentiation.

[0102] Specifically, the logically heterogeneous executors: achieve internal diversity by deploying control algorithms based on different theoretical foundations. Control theory systems with significant differences are selected, including classical control, adaptive control, fuzzy control, neural network control, and model predictive control, to build a diversified algorithm ecosystem. This diversity in theoretical foundations ensures that even if an attacker masters the vulnerabilities of a certain type of algorithm, it is difficult to break all executors at the same time, thus forming an effective security barrier at the algorithm level.

[0103] To strengthen the dynamic uncertainty of the system, the present invention designs a data set random sampling and algorithm adaptive selection mechanism. This mechanism first randomly samples the original control data to generate multiple differentiated data subsets, and then trains these data subsets with different control models. Through strict control accuracy evaluation, the system selects a combination of heterogeneous executors that meet the performance requirements, ensuring functional stability while enhancing the unpredictability of system behavior.

[0104] Spatio-temporally heterogeneous executors: enhance the diversity of executors in time and space dimensions. In the time dimension, different control periods and start times are assigned to each spatio-temporally heterogeneous executor, creating a gradient time structure. Through random delay mechanisms and dynamic period adjustment, the runtime sequence of spatio-temporally heterogeneous executors presents a high degree of randomness, effectively resisting security threats based on time sequence analysis and synchronization attacks.

[0105] In the spatial dimension, a distributed deployment strategy is adopted, and a spatial differentiation structure is established by carefully designing the logical positions and communication paths of the executors. The communication links between the executors introduce random disturbance factors, further enhancing the uncertainty of the system behavior. This spatio-temporal dual-heterogeneous design significantly improves the quasi-state defense capability of the system without increasing physical resources.

[0106] The dynamic management mechanism of the executor pool includes: the operation management of the executor pool adopts a dynamic selection and combination mechanism. According to the current security situation and operating state, the combination scheme of the activated executors is adjusted in real time, and the attack surface is formed by frequently changing the internal structure of the system. The multi-mode voter as the final decision unit comprehensively analyzes and weight fuses the outputs of multiple heterogeneous executors, ensuring that the system provides stable functions while maintaining high uncertainty of the internal structure.

[0107] Through the systematic combination of logical heterogeneity and spatio-temporal heterogeneity, the present application constructs a dynamically evolving heterogeneous executor pool, providing a solid infrastructure for the security protection of the wind-solar-fire-storage-hydrogen power grid, and realizing the quasi-state defense goal of "static function, dynamic structure".

[0108] Logical heterogeneity is realized by deploying multiple functionally equivalent but different implementation mechanism control modules on a unified hardware platform, including different control algorithms, different data processing procedures and different decision logic; spatio-temporal heterogeneity is realized by time sequence randomization, execution order disturbance and spatial deployment differentiation of the control modules, changing the space-time characteristics of the system to increase the difficulty of attack.

[0109] The present application realizes effective heterogeneity protection while maintaining engineering realizability by constructing an executor pool based on logical heterogeneity, spatio-temporal heterogeneity and interface heterogeneity. Logical heterogeneity is realized by deploying functionally equivalent but different implementation mechanism control modules on a unified hardware platform, avoiding the complexity and high cost of multiple hardware platforms. Spatio-temporal heterogeneity is realized by time sequence randomization and execution order disturbance, effectively changing the space-time characteristics of the system. This practical and feasible heterogeneity design provides effective security protection for the wind-solar-fire-storage-hydrogen power grid, significantly improving the system's resistance to composite attacks.

[0110] Each heterogeneous executor votes on the action space, and then the intelligent voting tally determines the target action space by weighting the voting results of each heterogeneous executor.

[0111] The intelligent voting tally adopts a dynamic weight distribution mechanism, which adjusts the voting weight in real time according to the reliability, performance and trustworthiness of each heterogeneous executor. The weight calculation formula considers reliability and performance to ensure the accuracy and reliability of the voting results.

[0112] The weight calculation of the intelligent voting tally is as follows: wherein, represents the weight of the executor at time ; , respectively represent the evaluation index of the executor in reliability and performance dimensions; , are weight coefficients.

[0113] The reliability index is calculated as follows: wherein, represents the reliability index of the executor; represents the number of failures of the executor within the statistical period, which is accumulated by monitoring events such as non-response, output error, and crash restart of the executor; represents the total number of running times, i.e., the total number of times the executor is called to execute tasks.

[0114] The performance index is calculated as follows: wherein, represents the performance index of the executor; represents the response speed index; represents the consistency index; , are weight coefficients.

[0115] The response speed index is calculated as follows: wherein, represents the actual response time of the executor, which is obtained by measuring the time interval from receiving threat information to outputting defense decisions; represents the reference response time, which is set as the standard response time required by the system.

[0116] The consistency index is calculated as follows: wherein, represents the output decision of the executor; represents the average value of the output of other executors, which is calculated by ; , respectively represent the maximum and minimum values of the decision output.

[0117] In the voting decision-making process, an anomaly detection mechanism is used to identify and isolate abnormal heterogeneous executors in a timely manner to prevent malicious output from affecting system decision-making. Specifically, the system deploys a multi-level anomaly detection mechanism to ensure decision-making quality. The system monitors the output of each heterogeneous executor in real time, establishes an anomaly identification model based on statistical analysis, and identifies potential anomalies by monitoring the deviation of the executor output from the group mean. When the output of an executor deviates from the group decision by more than twice the standard deviation, the system marks it as a suspected anomaly; when abnormal output occurs three times in a row, the system immediately isolates the executor and stops it from participating in subsequent voting processes. At the same time, the system establishes a behavior pattern analysis mechanism that can identify different types of anomaly patterns such as periodic attacks, sudden malicious output, and gradual performance degradation, and timely discover executors under attack or failure through pattern matching technology.

[0118] To ensure that the system can maintain normal protection functions in the case of partial executor failure, the system designs a perfect fault-tolerant mechanism that ensures the system can maintain normal protection functions in the case of partial executor failure. The system adopts the "two-thirds" principle to ensure that at least two-thirds of the total number of executors are in normal working condition. When the number of available executors approaches this threshold, the system automatically activates the redundant units in the standby executor pool. When some executors are isolated, the system immediately starts a weight redistribution algorithm to distribute the weight of the failed executors proportionally to the remaining normal executors, ensuring the continuity and stability of the voting results. In addition, the system also establishes a hierarchical degradation mechanism to prioritize core defense functions when the number of available executors is severely insufficient, and if necessary, use pre-set emergency defense strategies to maintain the most basic security protection.

[0119] The system continuously improves overall availability through real-time performance monitoring and adaptive optimization mechanisms. The monitoring system evaluates the confidence of the voting decision every second, and immediately issues a warning and starts the manual intervention process when the confidence is below the preset threshold. At the same time, the system records the detailed process of each anomaly detection and fault tolerance processing, and continuously optimizes the accuracy of anomaly detection and the effectiveness of fault tolerance strategies through machine learning algorithms. Through the synergistic effect of these innovative mechanisms, the system can maintain high defense decision-making accuracy in the face of complex attack environments, ensuring the safe and stable operation of the wind-solar-thermal-hydrogen power grid even when some components are attacked or fail, significantly improving the accuracy of voting decisions and effectively ensuring the overall availability of the system.

[0120] S104, achieving quasi-state defense of the wind-solar-thermal-hydrogen power grid through the target action space.

[0121] The target action space includes reconstructing the action space of the intelligent agent , wherein the action space of the intelligent agent represents the reconstruction type (logical reconstruction, space-time reconstruction, interface reconstruction, etc.), represents the reconstruction timing (periodic, event-triggered, hybrid-triggered, prediction-triggered, etc.), represents the reconstruction range (three levels of local, regional, and global), represents the reconstruction intensity (three levels of low, medium, and high); the action space of the execution agent wherein represents the activation state of the th execution agent, represents the weight distribution coefficient, represents the system redundancy level, represents the load balancing strategy; the action space of the voting agent wherein represents the voting rule (simple majority, weighted majority, dynamic rule), represents the exception handling strategy (elimination, weight reduction, correction), represents the fault-tolerant mechanism level, represents the arbitration algorithm selection; and the action space of the resource agent wherein represents the computing resource allocation vector, represents the communication resource allocation vector, represents the storage resource allocation vector, represents the network resource allocation vector. After the voting and voting of the logical heterogeneous and space-time heterogeneous execution agent pool, the final target action space is determined, which directly drives the wind-solar-thermal-hydrogen power grid to implement the mimic defense, and realizes the active protection mechanism of dynamic heterogeneous redundancy.

[0122] ​​In one embodiment, the multi-agent system follows a threat-driven parallel coordination mechanism, embodying the core features of mimicry defense multi-body execution, multi-mode decision. In the threat response stage, the threat situation assessment module simultaneously transmits threat information to the reconstruction agent and the voting agent, realizing parallel processing of threat information. The reconstruction agent autonomously determines the necessity of reconstruction and strategy parameters according to the threat level, type and impact range, and the voting agent synchronously receives threat level information and pre-adjusts voting rules and abnormal handling thresholds. In the strategy execution stage, the execution agent receives the reconstruction strategy, dynamically selects a combination of heterogeneous execution bodies and configures running parameters to form a diversified execution body pool. Each heterogeneous execution body processes defense tasks in parallel, and its output directly flows to the voting agent for real-time voting and result fusion. In the dynamic coordination stage, the resource agent continuously monitors the system running state, receives reconstruction resource requirements from the reconstruction agent, execution body resource requirements from the execution agent, and voting processing load requests from the voting agent, dynamically adjusts resource allocation strategies to ensure the performance stability of the system in the process of heterogeneous reconstruction. This coordination mechanism eliminates the bottleneck problem of traditional serial decision chain, realizes efficient mimicry defense response through parallel processing and dynamic feedback, and meets the real-time requirements of wind-solar-thermal-hydrogen power grid and the defense principle of dynamic heterogeneous redundancy.

[0123] In one embodiment, as shown in Figure 2 , first, an execution body pool based on logical heterogeneity and spatio-temporal heterogeneity is constructed, and multiple functionally equivalent but different implementation mechanism control modules are deployed on a unified software platform to provide a realistic and feasible heterogeneous basis for wind-solar-thermal-hydrogen power grid; then a special threat perception and environment monitoring module is deployed to accurately identify new energy power abnormalities, energy storage system malicious regulation, hydrogen production system safety threats, and multi-energy coordination abnormalities; then a mimicry defense decision model is constructed based on a multi-agent reinforcement learning framework, which divides the complex decision-making problem into a collaborative decision-making problem of multiple specialized agents, and realizes autonomous optimization of strategies through a hierarchical hybrid reward function; finally, a dynamic combination of heterogeneous execution bodies is selected according to the output strategy of the decision model, and the result is fused through an intelligent voting decider to realize adaptive reconstruction and intelligent protection of the system.

[0124] The present application forms a complete intelligent protection system by constructing a mimicry defense decision center, a heterogeneous execution body pool, an intelligent voting decider, a dynamic reconstruction manager, and a threat perception and environment monitoring module, etc. core components, and each component is coordinated through reinforcement learning algorithm.

[0125] The application constructs an adaptive, efficient and reliable security protection system, adopts a modular architecture design, realizes intelligent protection under the premise of ensuring the real-time requirement of the power grid, and has good engineering practicability and sustainable development capability. Through the hot plug and online upgrade mechanism of the heterogeneous executor pool, the system can easily integrate new control algorithms and software technologies; the threat detection module supports rapid learning and adaptation of new threat modes; the multi-agent reinforcement learning framework supports continuous evolution and optimization of strategies, ensuring that the system can dynamically expand and upgrade with the development of wind, light, fire, storage and hydrogen technologies and changes in the threat environment. At the same time, the threat detection and reconstruction decisions can meet the real-time requirements of the power system, and through the security constrained optimization mechanism, the protection decisions are ensured not to violate the basic safety requirements of the power grid operation, realizing the organic unity of security protection and system stability. The system availability is effectively guaranteed, which has good technical foresight and investment protection value, and also provides reliable protection for the safe and stable operation of the new power system.

[0126] In the application of the wind, light, fire, storage and hydrogen power grid mimic defense method based on reinforcement learning provided by the application, the wind, light, fire, storage and hydrogen power grid can be not monitored according to the real-time requirement of the power grid, and the wind, light, fire, storage and hydrogen power grid can be not monitored according to the real-time requirement of the power grid. Figure 1 The execution order of each step shown in the above embodiment is not limited, and the execution order of each step can be determined according to the actual situation.

[0127] The above is the wind, light, fire, storage and hydrogen power grid mimic defense method based on reinforcement learning provided by one or more embodiments of the application. Based on the same idea, the application also provides a corresponding wind, light, fire, storage and hydrogen power grid mimic defense system based on reinforcement learning. The system comprises: A threat perception and environment monitoring module, which monitors the wind, light, fire, storage and hydrogen power grid in real time, identifies new energy specific threats and multi-energy coordination abnormalities, and generates threat situation information of the wind, light, fire, storage and hydrogen power grid; A prediction module, which is used for taking the power grid operation state, threat situation, mimic defense state and historical performance of the wind, light, fire, storage and hydrogen power grid as a state space, and predicting the action space of the wind, light, fire, storage and hydrogen power grid through a mimic defense decision model; the mimic defense decision model adopts a multi-agent reinforcement learning framework, and decomposes the complex action space into multiple agent collaborative decisions; A voting module, which is used for voting the action space according to the executor pool with logic heterogeneity and space-time heterogeneity, and determining a target action space; A defense module, which is used for realizing mimic defense of the wind, light, fire, storage and hydrogen power grid through the target action space.

[0128] The specific limitations of the wind-solar-thermal-hydrogen power grid mimetic defense system based on reinforcement learning can refer to the limitations of the wind-solar-thermal-hydrogen power grid mimetic defense method based on reinforcement learning described above, and will not be repeated here. Each module in the wind-solar-thermal-hydrogen power grid mimetic defense system based on reinforcement learning described above can be realized by software, hardware and their combinations in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to call and execute the operations corresponding to each module by the processor.

[0129] The application further provides a computer readable storage medium, which stores a computer program, and the computer program can be used to execute the wind-solar-thermal-hydrogen power grid mimetic defense method based on reinforcement learning. Figure 1 The application provides a wind-solar-thermal-hydrogen power grid mimetic defense method based on reinforcement learning.

[0130] The application further provides a computer readable storage medium, which stores a computer program, and the computer program can be used to execute the wind-solar-thermal-hydrogen power grid mimetic defense method based on reinforcement learning. Figure 3 The structure diagram of the computer device is shown in the figure, and the computer device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory. Figure 3 As shown in the figure, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and of course can also include other hardware required by the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to realize the wind-solar-thermal-hydrogen power grid mimetic defense method based on reinforcement learning provided above. Figure 1 The application provides a wind-solar-thermal-hydrogen power grid mimetic defense method based on reinforcement learning.

[0131] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium and can include the processes of the above-mentioned embodiments when executed. Any reference to a memory, storage, database or other medium used in the embodiments provided by the application can include at least one of a non-volatile and volatile memory. The non-volatile memory can include a read-only memory (ROM), a tape, a floppy disk, a flash memory or an optical memory. The volatile memory can include a random access memory (RAM) or an external cache memory. As an illustration but not limitation, the RAM can be in various forms such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0132] The technical features of the above embodiments can be combined in any manner. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not contradict each other, they should be considered to be within the scope of the present application.

Claims

1. A reinforcement learning-based method for mimicking the defense of a wind-solar-thermal-storage-hydrogen power grid, characterized in that, include: Real-time monitoring of the wind, solar, thermal, energy storage, and hydrogen power grid; identification of unique threats to new energy sources and anomalies in multi-energy coordination; generation of threat situation information for the wind, solar, thermal, energy storage, and hydrogen power grid. Using the grid operation status, threat situation, mimicry defense status, and historical performance of the wind-solar-thermal-storage-hydrogen grid as the state space, the action space of the wind-solar-thermal-storage-hydrogen grid is predicted through the mimicry defense decision model. The mimicry defense decision model adopts a multi-agent reinforcement learning framework to decompose the complex action space into collaborative decision-making by multiple agents. The target action space is determined by voting on the execution body pool based on logical heterogeneity and spatiotemporal heterogeneity. Mimicry defense against wind, solar, thermal, energy storage, and hydrogen power grids is achieved through the target action space.

2. The method according to claim 1, characterized in that, Real-time monitoring of the wind-solar-thermal-storage-hydrogen power grid is conducted to identify unique threats to new energy sources and anomalies in multi-energy coordination, generating threat situation information for the wind-solar-thermal-storage-hydrogen power grid, including: Anomaly identification is performed on various equipment in the wind, solar, thermal, energy storage, and hydrogen power grid. Based on the anomaly identification results, the threat situation information of the wind, solar, thermal, energy storage, and hydrogen power grid is determined.

3. The method according to claim 2, characterized in that, The wind-solar-thermal-storage-hydrogen grid comprises various equipment including wind power systems, photovoltaic systems, energy storage systems, and hydrogen production systems; anomaly identification is performed on each piece of equipment in the wind-solar-thermal-storage-hydrogen grid, including: Anomaly feature spaces are constructed using wind speed-power relationship models and irradiance-power relationship models. High-dimensional statistical properties and time-series dynamic evolution laws are used to identify and classify power output anomalies in wind power systems and photovoltaic systems. By using a pre-built SOC state evolution trajectory model and a feature library of charging and discharging behavior patterns, and through multi-scale state monitoring and behavior consistency verification, malicious regulation attacks and unauthorized operations on energy storage systems can be identified. By dynamically evaluating electrolysis efficiency and analyzing the propagation of process parameter anomalies, and combining gas quality characterization and safety threshold constraints, safety threats to hydrogen production systems are identified. By conducting dynamic power balance assessment, frequency response coordination verification, and voltage stability analysis, abnormal modes of coordinated operation among wind, solar, thermal, and hydrogen storage systems are identified.

4. The method according to claim 3, characterized in that, Threat situation information for wind, solar, thermal, energy storage, and hydrogen power grids includes threat level, attack type, scope of impact, and duration; The power grid operating status includes the power generation status of various equipment, system frequency and voltage parameters, and equipment health status. The mimicry defense status includes the current execution configuration, reconstruction history, voting weight distribution, and system load; Historical performance metrics include protection success rate, false alarm rate, system availability, and response time.

5. The method according to claim 1, characterized in that, Multi-agent reinforcement learning frameworks include: Reconstructing the intelligent agent : Responsible for making decisions on mimicry reconstruction strategies, its action space is ,in, Indicates the refactoring type. Indicates the timing of the restructuring. Indicates the scope of the reconstruction. Indicates the reconstruction intensity; Execution agent : Responsible for managing heterogeneous execution entities, its action space is ,in, Indicates the first i The activation state of each execution entity Indicates the weighting coefficient. Indicates the system redundancy level. Indicates the load balancing strategy; Voting Intelligent Agent : Responsible for optimizing voting strategies, its action space is ,in, Indicate the voting rules, Indicates the exception handling strategy. Indicates the level of fault tolerance mechanism. Indicates the choice of arbitration algorithm; Resource Intelligent Agent : Responsible for coordinating and allocating system resources; its action space is ,in, Represents the computational resource allocation vector. Represents a communication resource allocation vector. Represents a storage resource allocation vector. This represents the network resource allocation vector.

6. The method according to claim 5, characterized in that, The multi-agent reinforcement learning framework employs a hierarchical hybrid reward function structure: in, Represents intelligent agents i Comprehensive rewards, , , , and These represent the local reward evaluations of the intelligent agent in the dimensions of security, reliability, efficiency, adaptability, and cost, respectively. For globally shared reward signals; , , , , and For intelligent agents i Weighting coefficients; The specific calculation methods for rewards in each dimension are as follows: Security rewards The formula used to comprehensively evaluate the effectiveness of threat defense is as follows: in, The attack success rate is calculated as the ratio of the number of successful intrusions to the total number of attacks within a statistical time window. Indicates the accuracy of threat detection; Indicates the false alarm rate; , , These are weighting coefficients; Reliability Bonus The formula used to assess system stability is as follows: in, Indicates the system's available time. Indicates the total running time; Indicates the number of failures; Indicates the fault recovery time; and This is the penalty coefficient; Efficiency rewards The formula used to evaluate decision response performance is as follows: in, This represents the standard response time baseline value. Indicates the actual response time; Indicates resource utilization rate; This is the resource consumption penalty coefficient; Adaptive rewards The formula used to assess the ability to adjust strategies is as follows: in, Indicates the extent of performance improvement. Indicates changes in threat intensity; Indicates the time taken to adjust the strategy; This is the time sensitivity coefficient; Cost Incentive The formula used to assess defense costs is as follows: in, The computational cost is represented by CPU utilization and power consumption. The communication cost is represented by network bandwidth usage; This represents the refactoring cost, including the overhead of switching execution units and updating configurations; Globally shared rewards The formula used to evaluate the overall protection effectiveness of the system is: in, It represents the overall system loss, calculated comprehensively based on grid frequency deviation, voltage over-limit, and load loss. This represents a score indicating the effectiveness of agent collaboration. Indicates the number of decision-making conflicts; , , These are the weighting coefficients.

7. The method according to claim 1, characterized in that, Construct an execution pool that is logically and spatiotemporally heterogeneous, including: This involves deploying multiple functionally equivalent but differently implemented logically heterogeneous executors on a unified hardware platform; these heterogeneous executors include different control algorithms, different data processing flows, and different decision-making logics. Multiple spatiotemporally heterogeneous execution entities are constructed by temporal randomization, execution order perturbation, and spatial deployment differentiation.

Images