Plug-in function and online test method for pre-training model of power system

Abstract

The invention discloses a plug-in function and online test method for a pre-training model of a power system, and designs a pre-training model oriented to the power industry by combining NLP, BERT and reinforcement learning technologies, and realizes a data-driven intelligent control strategy. Through semi-supervised learning, labeled data and unlabeled data are integrated, the generalization ability of the model is improved, and a high-quality initialization model is provided for reinforcement learning agents. The pre-training model is disassembled into independent plug-in modules, and the plug-in modules are integrated with a simulation platform through an API, so that on-demand loading and scene switching of the modules are realized, and the flexibility of the system is ensured. A dynamic event and a system fault are simulated on an HIL platform, and a reinforcement learning agent is pre-trained in combination with an MPC controller, so that the strategy optimization process can continuously adapt to a system state. In addition, model parameters and control strategies are continuously updated through a particle swarm optimization algorithm, and it is ensured that the RL agent and the plug-in module are always kept synchronous with system requirements.

Description

Technical Field

[0001] The present invention belongs to the field of plug-in testing, and in particular, relates to a plug-in function and an online testing method of a pre-trained model of an electric power system. Background Art

[0002] As the scale and complexity of power systems increase, traditional control strategies are difficult to adapt to real-time changes and scheduling requirements in complex scenarios. Reinforcement learning (RL), as a data-driven control method, can dynamically adapt to changes in system states, but its effectiveness depends on high-quality model parameters and strategy optimization. At the same time, MPC controllers have been widely used in power systems due to their control capabilities under prediction and constraint conditions. However, how to effectively combine MPC and RL technologies and deeply integrate them with the HIL simulation platform to achieve optimization of control strategies in multiple scenarios is a key challenge in the current intelligent power system. In addition, real-time data collection and optimization, dynamic update of model parameters, on-demand loading of plug-in modules, and closed-loop control capabilities of the simulation environment all put forward higher requirements on the stability and responsiveness of the power system.

[0003] In view of this, the present invention is proposed. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art, provide a plug-in function and an online testing method for a power system pre-training model, and solve the problems raised in the above-mentioned background technology.

[0005] In order to solve the above technical problems, the basic concept of the technical solution adopted by the present invention is:

[0006] A plug-in function and online testing method for a power system pre-training model, comprising the following steps:

[0007] Step S1: Use NLP, BERT, and reinforcement learning techniques to combine power industry data and standard documents to train the pre-trained model, and integrate labeled and unlabeled data through semi-supervised learning;

[0008] Step S2: Split the pre-trained large model into independent plug-in modules, integrate them with the simulation platform through API, and load them on demand according to different power scenarios;

[0009] Step S3: Integrate the plug-in module and physical devices on the HIL platform, and simulate dynamic events and system failures through real-time simulation;

[0010] Step S4: Pre-train the reinforcement learning agent using the MPC controller in the HIL simulation to make it imitate the behavior of the MPC and gradually optimize the control strategy;

[0011] Step S5: Collect real-time data from HIL simulation and system operation, and update the reinforcement learning agent model parameters and control strategy through the optimization algorithm, and feed the data back to the RL agent and plug-in module to adjust its behavior.

[0012] Optionally, use NLP, BERT, and reinforcement learning techniques, combined with power industry data and standard documents, to train a pre-trained model, and integrate labeled and unlabeled data through semi-supervised learning. The steps are:

[0013] S101: Collect unlabeled data from operation logs, historical data, and technical standards, and obtain labeled data from maintenance records and fault reports. Then, clean, standardize, and normalize the acquired data and convert it into an input format acceptable to the BERT model.

[0014] S102: Input the preprocessed unlabeled data into the BERT model, randomly mask the words and predict the original content through the MLM task, and judge the logical relationship between the texts through the NSP task, then use the Adam optimizer and perform distributed training through the GPU cluster to generate the initial BERT model weights;

[0015] S103: Use the BERT model obtained by unsupervised pre-training to generate pseudo labels for unlabeled data, and integrate the pseudo labels with labeled data into an extended dataset;

[0016] S104: Fine-tune the BERT model on the extended dataset, optimize the prediction of labeled data through cross-entropy loss, and use pseudo-label loss to adjust the prediction results of unlabeled data. The fine-tuned BERT model is used as the initial model of the reinforcement learning agent.

[0017] Optionally, the steps to split the pre-trained large model into independent plug-in modules and integrate the simulation platform through the API are:

[0018] S201: According to different power scenarios, the functions of the pre-trained large model are decomposed into independent plug-in modules. Then, each plug-in module is mapped to a specific task. Subsequently, each plug-in module is encapsulated using a microservice architecture, and an API interface is designed for it, defining the input and output formats.

[0019] S202: Configure the API address in the MATLAB / Simulink platform and write a calling script to pass the input data of the simulation platform to the plug-in module and receive the prediction results or control instructions returned by the module;

[0020] S203: Dynamically load and switch plug-in modules according to different power scenarios and real-time data. Finally, synchronize the running results of the plug-in modules with the data of the simulation platform, and use load balancing algorithms and multi-threaded processing to ensure that there is no delay in module switching.

[0021] Optionally, the steps of integrating the plug-in module and the physical device on the HIL platform and simulating dynamic events and system failures through real-time simulation are:

[0022] S301: Establish communication between the split plug-in module and the HIL platform through the API, and then configure the API address to ensure that the plug-in module can receive data in real time and return control instructions;

[0023] S302: Build a preset simulation scenario on the HIL platform, then connect the physical device to the HIL platform to synchronize data transmission between the device and the system, then pass the data collected in real time by the simulation platform to the plug-in module, and receive the control instructions output by it;

[0024] S303: During the simulation process, the system status is monitored in real time through the HIL platform, dynamic events and system faults are detected, and the detected events and fault data are transmitted to the plug-in module during the detection, which triggers the control logic to generate corresponding control instructions, and transmits the instructions back to the HIL system;

[0025] S304: using the adaptive module of the FPGA to automatically update the control parameters, and optimizing the parameters in real time based on the steepest descent algorithm;

[0026] S305: The input, output and event logs during the simulation process are transmitted to the central monitoring system, and synchronized with the results of the HIL platform to record the status and control instructions of each API call.

[0027] Optionally, the steps to pre-train a reinforcement learning agent using an MPC controller in a HIL simulation so that it mimics the behavior of the MPC and gradually optimizes the control strategy are:

[0028] S401: Build an MPC controller model in the HIL simulation platform, and define the control time domain and prediction time domain based on the state space model of the system. Then, solve the optimal control instruction through the quadratic programming algorithm at each time step, and use its output as the benchmark control strategy;

[0029] S402: running the MPC controller in the HIL platform, collecting system states and corresponding control instructions under different working conditions and dynamic events, recording state-action pair data, and storing them in a training data set as training data for the reinforcement learning agent;

[0030] S403: Deploy the pre-trained reinforcement learning agent in the HIL simulation to receive system status and feedback data in real time. During the data reception process, the agent uses the Proximal Policy Optimization algorithm to continuously update the strategy and adaptively optimize its control behavior based on the real-time reward signal.

[0031] S404: After each round of online training, evaluate the control strategy of the reinforcement learning agent and compare it with the baseline control strategy of the MPC, adjust the hyperparameters of the RL agent according to the evaluation results, and continue online training.

[0032] Optionally, an MPC controller model is built in the HIL simulation platform, and the control time domain and prediction time domain are defined based on the state space model of the system. Then, the optimal control instruction is solved by the quadratic programming algorithm at each time step, and its output is used as the benchmark control strategy in the following steps:

[0033] Based on the operating characteristics of the power system, a state space model X(t+1)=AX(t)+BU(t)X(t+1) is established, where X(t) represents the system state, U(t) represents the control input, and A and B are the state and control matrices, respectively;

[0034] Set the control time domain N c and prediction time domain N p , where N c Determine the number of steps, N, to apply the control command p Determine the prediction length of the future state of the system, and then, at each time step t, construct the objective function based on the current state X(t): Among them, Q and R are weight matrices, indicating the optimization priority of state and control input. Then, set the system constraint condition as U min ≤U(t)≤U max and status restrictions;

[0035] At each time step, the optimal control input sequence U is solved by the quadratic programming algorithm. * ={U(t),U(t+1),...,U(t+N c -1)}, only the first control instruction U(t) in the sequence is applied to the system, and the rest is used for the next iterative update;

[0036] Finally, the optimal control instruction U(t) after each round of solution is used as the benchmark control strategy, and its result is passed to the RL agent in the HIL simulation platform.

[0037] Optionally, collect real-time data from HIL simulation and system operation, and update model parameters and control strategies through optimization algorithms, and feed data back to RL agents and plug-in modules to adjust their behavior as follows:

[0038] S501: Collecting real-time data from the HIL simulation platform and the system during operation and transmitting the collected data to a database through an API interface for storage;

[0039] S502: Clean the collected real-time data, use a Kalman filter to smooth the noise data, detect and remove missing values ​​and abnormal data, and normalize the data into a format acceptable to the model;

[0040] S503: Input the preprocessed data into the optimization algorithm, update the reinforcement learning agent model parameters through the particle swarm optimization algorithm, and adjust the control strategy in real time to adapt to the current system state.

[0041] Optionally, the steps of updating the model parameters and adjusting the control strategy in real time by using the particle swarm optimization algorithm are:

[0042] S5031: Initialize the particle swarm, and the position of each particle represents the model parameter X i =(x i1 , x i2 , dots, x id ), where i is the particle number and d is the parameter dimension. Then, initialize the velocity of each particle to V i =(v i1 , v i2 ,dots,v id ), and obtain the optimal historical solution P of the particle individual i And the global optimal solution G, then define the fitness function f(X i ) Evaluate the performance of each particle;

[0043] S5032: In each iteration, update the particle's velocity and position. The velocity update formula is V i (t+1)=ωV i (t)+c 1 r 1 (P i -X i (t))+c 2 r 2 (GX i (t)), where ω is the inertia weight, c 1 , c 2 is the learning factor, r 1 , r 2 is a random number (0, 1), and the position update formula is: i (t+1)=X i (t)+V i (t+1), then, using the fitness function f(X i ) evaluates the new position of each particle. If f(X i (t+1))>f(P i ), then update the individual optimal solution P i =X i(t+1). If the fitness of a particle is better than the current global optimal solution G, then update the global optimal solution G = X i (t+1);

[0044] S5033: The global optimal solution G is used as the latest parameter of the model, and the model is updated in real time through the API interface. The control strategy is generated according to the new parameters. The expression is: U(t) = K·X, where U(t) is the control instruction, K is the control matrix, and X(t) is the system state. Then, the feedback data is input into the particle swarm optimization algorithm, and the fitness f(X) is re-evaluated with the new state data. i ), and start the next round of optimization, repeating steps S5032 to S5035 to ensure that the reinforcement learning agent model continuously updates parameters and control strategies according to the real-time status.

[0045] After adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art. Of course, any product implementing the present invention does not necessarily need to achieve all the advantages described below at the same time:

[0046] This application combines NLP, BERT and reinforcement learning technology to design a pre-trained model for the power industry and implements a data-driven intelligent control strategy. By integrating labeled and unlabeled data through semi-supervised learning, the generalization ability of the model is improved, and a high-quality initialization model is provided for the reinforcement learning agent. The pre-trained model is disassembled into independent plug-in modules and integrated with the simulation platform through API to achieve on-demand loading and scene switching of modules, ensuring the flexibility of the system. Dynamic events and system failures are simulated on the HIL platform, and the reinforcement learning agent is pre-trained in combination with the MPC controller, so that its strategy optimization process can continuously adapt to the system state. In addition, the model parameters and control strategies are continuously updated through the particle swarm optimization algorithm to ensure that the RL agent and plug-in module are always synchronized with system requirements.

[0047] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described below are only some embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0049] In the figure:

[0050] Figure 1 Flowchart of the plug-in function and online testing method for the pre-trained model of the force system.

[0051] It should be noted that these drawings and textual descriptions are not intended to limit the conceptual scope of the present invention in any way, but are intended to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0052] The present invention will now be described in further detail with reference to the accompanying drawings.

[0053] See also Figure 1 As shown, in this embodiment, a plug-in function and an online testing method of a power system pre-training model are provided, including the following steps:

[0054] Step S1: Use NLP, BERT, and reinforcement learning techniques to combine power industry data and standard documents to train the pre-trained model, and integrate labeled and unlabeled data through semi-supervised learning;

[0055] Step S2: Split the pre-trained large model into independent plug-in modules, integrate them with the simulation platform through API, and load them on demand according to different power scenarios;

[0056] Step S3: Integrate the plug-in module and physical devices on the HIL platform, and simulate dynamic events and system failures through real-time simulation;

[0057] Step S4: Pre-train the reinforcement learning agent using the MPC controller in the HIL simulation to make it imitate the behavior of the MPC and gradually optimize the control strategy;

[0058] Step S5: Collect real-time data from HIL simulation and system operation, and update the reinforcement learning agent model parameters and control strategy through the optimization algorithm, and feed the data back to the RL agent and plug-in module to adjust its behavior.

[0059] In this embodiment, NLP, BERT and reinforcement learning techniques are used to combine power industry data and standard documents to train a pre-trained model, and the steps of integrating labeled and unlabeled data through semi-supervised learning are as follows:

[0060] S101: Collect unlabeled data from operation logs, historical data, and technical standards, and obtain labeled data from maintenance records and fault reports. Then, clean, standardize, and normalize the acquired data and convert it into an input format acceptable to the BERT model; unlabeled data comes from the daily operation data of the power system, real-time monitoring logs, and technical standard documents. These data do not come with specific labels, such as equipment status or operating instructions. Labeled data comes from equipment fault reports, maintenance records, and manually labeled data sets, and contains clear label information;

[0061] S102: Input the preprocessed unlabeled data into the BERT model, randomly mask the words and predict the original content through the MLM task, and judge the logical relationship between the texts through the NSP task, then use the Adam optimizer and perform distributed training through the GPU cluster to generate the initial BERT model weights;

[0062] S103: Use the BERT model obtained by unsupervised pre-training to generate pseudo labels for unlabeled data, and integrate the pseudo labels with labeled data into an extended dataset;

[0063] S104: Fine-tune the BERT model on the extended dataset, optimize the prediction of labeled data through cross-entropy loss, and use pseudo-label loss to adjust the prediction results of unlabeled data. The fine-tuned BERT model is used as the initial model of the reinforcement learning agent.

[0064] In this embodiment, the steps of splitting the pre-trained large model into independent plug-in modules and integrating the simulation platform through the API are as follows:

[0065] S201: According to different power scenarios (such as frequency control, load management, and equipment maintenance), the functions of the pre-trained large model are decomposed into independent plug-in modules. Then, each plug-in module is mapped to a specific task. Subsequently, each plug-in module is encapsulated using a microservice architecture, and an API interface is designed for it, defining the input and output formats (such as JSON or XML).

[0066] S202: Configure the API address in the MATLAB / Simulink platform and write a calling script to pass the input data of the simulation platform to the plug-in module and receive the prediction results or control instructions returned by the module.

[0067] S203: Dynamically load and switch plug-in modules according to different power scenarios and real-time data. Finally, synchronize the running results of the plug-in modules with the data of the simulation platform, and use load balancing algorithms and multi-threaded processing to ensure that there is no delay in module switching.

[0068] In this embodiment, the steps of integrating the plug-in module and the physical device on the HIL platform and simulating dynamic events and system failures through real-time simulation are as follows:

[0069] S301: The split plug-in module establishes communication with the HIL platform through the API, and then configures the API address to ensure that the plug-in module can receive data in real time and return control instructions.

[0070] S302: Build preset simulation scenarios on the HIL platform, including normal operation status, load changes and system failures. Then, connect the physical device to the HIL platform to synchronize data transmission between the device and the system. Then, pass the data collected in real time by the simulation platform to the plug-in module and receive the control instructions output by it.

[0071] S303: During the simulation process, the system status is monitored in real time through the HIL platform, dynamic events and system faults are detected, and the detected events and fault data are transmitted to the plug-in module during the detection, which triggers the control logic to generate corresponding control instructions, and transmits the instructions back to the HIL system;

[0072] S304: using the adaptive module of the FPGA to automatically update the control parameters, and optimizing the parameters in real time based on the steepest descent algorithm;

[0073] S305: The input, output and event logs during the simulation process are transmitted to the central monitoring system, and synchronized with the results of the HIL platform to record the status and control instructions of each API call.

[0074] In this embodiment, the steps of using the MPC controller to pre-train the reinforcement learning agent in the HIL simulation so that it imitates the behavior of the MPC and gradually optimizes the control strategy are:

[0075] S401: Build an MPC controller model in the HIL simulation platform, and define the control time domain and prediction time domain based on the state space model of the system. Then, solve the optimal control instruction through the quadratic programming algorithm at each time step, and use its output as the benchmark control strategy;

[0076] S402: running the MPC controller in the HIL platform, collecting system states and corresponding control instructions under different working conditions and dynamic events, recording state-action pair data, and storing them in a training data set as training data for the reinforcement learning agent;

[0077] S403: Deploy the pre-trained reinforcement learning agent in the HIL simulation to receive system status and feedback data in real time. During the data reception process, the agent uses the Proximal Policy Optimization algorithm to continuously update the strategy and adaptively optimize its control behavior based on the real-time reward signal.

[0078] S404: After each round of online training, evaluate the control strategy of the reinforcement learning agent and compare it with the baseline control strategy of the MPC, adjust the hyperparameters of the RL agent according to the evaluation results, and continue online training.

[0079] In this embodiment, an MPC controller model is built in the HIL simulation platform, and the control time domain and the prediction time domain are defined based on the state space model of the system. Then, the optimal control instruction is solved by the quadratic programming algorithm at each time step, and the output is used as the benchmark control strategy. The steps are:

[0080] Based on the operating characteristics of the power system, a state space model X(t+1)=AX(t)+BU(t)X(t+1) is established, where X(t) represents the system state, U(t) represents the control input, and A and B are the state and control matrices, respectively;

[0081] Set the control time domain N c and prediction time domain N p , where N c Determine the number of steps, N, to apply the control command p Determine the prediction length of the future state of the system, and then, at each time step t, construct the objective function based on the current state X(t): Among them, Q and R are weight matrices, indicating the optimization priority of state and control input. Then, set the system constraint condition as U min ≤U(t)≤U max and status restrictions;

[0082] At each time step, the optimal control input sequence U is solved by the quadratic programming algorithm. * ={U(t),U(t+1),...,U(t+N c -1)}, only the first control instruction U(t) in the sequence is applied to the system, and the rest is used for the next iterative update;

[0083] Finally, the optimal control instruction U(t) after each round of solution is used as the benchmark control strategy, and its result is passed to the RL agent in the HIL simulation platform.

[0084] This embodiment achieves precise control of the power system by building an MPC controller model on the HIL simulation platform and defining the control time domain and prediction time domain based on the state space model of the system. The quadratic programming algorithm is used to solve the optimal control input sequence at each time step, ensuring the efficient operation of the system under the constraints. By setting the weight matrix of the state and control input, the priority coordination between different control objectives is ensured. In addition, only the first control instruction of the current time step is applied, and the subsequent instructions are left for the next iteration, which enhances the flexibility and real-time performance of the control. The MPC control strategy, as a benchmark, provides a stable initial strategy for the reinforcement learning agent, enabling the RL agent to gradually optimize the control behavior in the simulation platform to ensure the responsiveness and robustness of the system in a dynamic environment. This control method based on the combination of MPC and RL realizes the organic integration of accurate prediction and optimal control, and improves the stability and adaptability of the power system in complex scenarios.

[0085] In this embodiment, the steps of collecting real-time data from HIL simulation and system operation, updating model parameters and control strategies through optimization algorithms, and feeding the data back to the RL agent and plug-in module to adjust their behavior are as follows:

[0086] S501: Collecting real-time data from the HIL simulation platform and the system during operation and transmitting the collected data to a database through an API interface for storage;

[0087] S502: Clean the collected real-time data, use a Kalman filter to smooth the noise data, detect and remove missing values ​​and abnormal data, and normalize the data into a format acceptable to the model;

[0088] S503: Input the preprocessed data into the optimization algorithm, update the reinforcement learning agent model parameters through the particle swarm optimization algorithm, and adjust the control strategy in real time to adapt to the current system state.

[0089] In this embodiment, the steps of updating the model parameters and adjusting the control strategy in real time by using the particle swarm optimization algorithm are as follows:

[0090] S5031: Initialize the particle swarm, and the position of each particle represents the model parameter X i =(x i1 , x i2 , dots, x id ), where i is the particle number and d is the parameter dimension. Then, initialize the velocity of each particle to V i =(v i1 , v i2 , dots, v id ), and obtain the optimal historical solution P of the particle individual i And the global optimal solution G, then define the fitness function f(Xi ) Evaluate the performance of each particle;

[0091] S5032: In each iteration, update the particle's velocity and position. The velocity update formula is V i (t+1)=ωV i (t)+c 1 r 1 (P i -X i (t))+c 2 r 2 (GX i (t)), where ω is the inertia weight, c 1 , c 2 is the learning factor, r 1 , r 2 is a random number (0, 1), and the position update formula is: i (t+1)=X i (t)+V i (t+1), then, using the fitness function f(X i ) evaluates the new position of each particle. If f(X i (t+1))>f(P i ), then update the individual optimal solution P i =Xx(t+1). If the fitness of a particle is better than the current global optimal solution G, then the global optimal solution G=X i (t+1);

[0092] S5033: The global optimal solution G is used as the latest parameter of the model, and the model is updated in real time through the API interface. The control strategy is generated according to the new parameters. The expression is: U(t) = K·X, where U(t) is the control instruction, K is the control matrix, and X(t) is the system state. Then, the feedback data is input into the particle swarm optimization algorithm, and the fitness f(X) is re-evaluated with the new state data. i ), and start the next round of optimization, repeating steps S5032 to S5035 to ensure that the reinforcement learning agent model continuously updates parameters and control strategies according to the real-time status.

[0093] It should be noted that the method of updating model parameters and adjusting control strategies in real time through the particle swarm optimization algorithm ensures that the model can dynamically adapt to the real-time state changes of the power system and ensure the continuous optimization of the control strategy. By initializing the particle swarm, defining the fitness function, and iteratively updating the position and velocity of the particles in each round, the real-time discovery of the global optimal solution is ensured. In each round of iteration, the global optimal solution is used as the latest parameter of the model using the secondary optimization results, and the parameters are applied to the system control strategy in real time through the API interface to ensure that the strategy can be adjusted in line with the changes in the system state. In addition, by looping the feedback data into the optimization algorithm, the reinforcement learning agent can continuously update the model parameters and control strategies to avoid system rigidity and ensure its stability and accuracy in multi-scenario operation. This dynamic adaptive optimization mechanism improves the decision-making efficiency of the model in complex power scenarios and reduces control delays.

[0094] Example 1: In this example, a power system pre-training model was built based on NLP, BERT and reinforcement learning technology, and it was split into plug-in modules, and its performance was tested through the HIL simulation platform. The experiment verified the functions of each module in different power scenarios, and used the MPC controller as the initial strategy of the reinforcement learning agent for optimization.

[0095] The BERT model was unsupervised pre-trained on a GPU cluster using 50GB of data collected from historical power system operation logs, monitoring data, and fault reports, 10% of which was labeled data, and trained for 48 hours using the Adam optimizer (learning rate of 5e-5). After training, pseudo labels were generated for the unlabeled data and combined with the labeled data into an extended dataset, which reached 70GB, and was fine-tuned to form the initial model of the reinforcement learning agent.

[0096] The pre-trained model is split into five plug-in modules for prediction, alarm, control, policy management, and data synchronization. Each module is integrated into the MATLAB / Simulink simulation platform through the RESTful API, and five power scenarios such as load change and line short circuit are built on the platform. The prediction time domain (20 steps) and control time domain (5 steps) are defined through the MPC controller in the HIL platform, and voltage, current, and frequency data are collected under different working conditions to generate 5,000 state-action pairs of data to provide initial training data for the RL agent.

[0097] The reinforcement learning agent is run on the HIL platform, and the PPO algorithm is used for online training for 10 hours, and real-time feedback is achieved through the API interface. The particle swarm optimization algorithm (PSO) is used to optimize the model parameters of the RL agent, iteratively update the control strategy once a second, and adjust the agent behavior in real time according to the changes in the system state to ensure the adaptability of the strategy optimization and system response.

[0098] Experiments show that the response speed of the control strategy after reinforcement learning optimization is significantly improved compared with that before optimization. In dynamic scenarios such as load mutation and equipment failure, the response time is shortened by more than 40% on average, and the control error is reduced from 5%-8% of the MPC strategy to 2%-3% after RL optimization. For example, in the load mutation scenario, the optimized response time is reduced to 2.1 seconds, which is more advantageous than the 3.5 seconds of MPC. In addition, in the line short circuit scenario, the control error of the RL agent is reduced from 7.1% of the MPC strategy to 3.5%. Through multi-threading and load balancing, the system realizes delay-free switching and efficient operation of plug-in modules, ensuring data synchronization and real-time strategy updates during the simulation process. The overall results show that this method shows good stability and adaptability when dealing with complex power system scenarios.

[0099] The present invention is not limited to the above-mentioned embodiments. Anyone should be aware that any structural changes made under the enlightenment of the present invention, and any technical solutions that are the same or similar to the present invention, fall within the protection scope of the present invention. The technology, shape, and structural parts not described in detail in the present invention are all well-known technologies.

Claims

1. A plug-in function and online testing method for a power system pre-training model, characterized in that: The following steps are involved: Step S1: Use NLP, BERT, and reinforcement learning techniques to combine power industry data and standard documents to train the pre-trained model, and integrate labeled and unlabeled data through semi-supervised learning; Step S2: Split the pre-trained large model into independent plug-in modules, integrate them with the simulation platform through API, and load them on demand according to different power scenarios; Step S3: Integrate the plug-in module and physical devices on the HIL platform, and simulate dynamic events and system failures through real-time simulation; Step S4: Pre-train the reinforcement learning agent using the MPC controller in the HIL simulation to make it imitate the behavior of the MPC and gradually optimize the control strategy; Step S5: Collect real-time data from HIL simulation and system operation, and update the reinforcement learning agent model parameters and control strategy through the optimization algorithm, and feed the data back to the RL agent and plug-in module to adjust its behavior.

2. The plug-in function and online testing method of a power system pre-training model according to claim 1 is characterized in that: Using NLP, BERT and reinforcement learning techniques, combined with power industry data and standard documents, the pre-trained model is trained, and the steps to integrate labeled and unlabeled data through semi-supervised learning are as follows: S101: Collect unlabeled data from operation logs, historical data, and technical standards, and obtain labeled data from maintenance records and fault reports. Then, clean, standardize, and normalize the acquired data and convert it into an input format acceptable to the BERT model. S102: Input the preprocessed unlabeled data into the BERT model, randomly mask the words and predict the original content through the MLM task, and judge the logical relationship between the texts through the NSP task, then use the Adam optimizer and perform distributed training through the GPU cluster to generate the initial BERT model weights; S103: Use the BERT model obtained by unsupervised pre-training to generate pseudo labels for unlabeled data, and integrate the pseudo labels with labeled data into an extended dataset; S104: Fine-tune the BERT model on the extended dataset, optimize the prediction of labeled data through cross-entropy loss, and adjust the prediction results of unlabeled data using pseudo-label loss. The fine-tuned BERT model is used as the initial model of the reinforcement learning agent.

3. The plug-in function and online testing method of a power system pre-training model according to claim 1 is characterized in that: The steps to split the pre-trained large model into independent plug-in modules and integrate the simulation platform through the API are: S201: According to different power scenarios, the functions of the pre-trained large model are decomposed into independent plug-in modules. Then, each plug-in module is mapped to a specific task. Subsequently, each plug-in module is encapsulated using a microservice architecture, and an API interface is designed for it, defining the input and output formats. S202: configuring the API address in the simulation platform, and writing a calling script to pass the input data of the simulation platform to the plug-in module, and receiving the prediction results or control instructions returned by the module; S203: Dynamically load and switch plug-in modules according to different power scenarios and real-time data. Finally, synchronize the running results of the plug-in modules with the data of the simulation platform, and use load balancing algorithms and multi-threaded processing to ensure that there is no delay in module switching.

4. The plug-in function and online testing method of a power system pre-training model according to claim 1, characterized in that: The steps to integrate plug-in modules and physical devices on the HIL platform and simulate dynamic events and system failures through real-time simulation are: S301: Establish communication between the split plug-in module and the HIL platform through the API, and then configure the API address to ensure that the plug-in module can receive data in real time and return control instructions; S302: Build a preset simulation scenario on the HIL platform, then connect the physical device to the HIL platform to synchronize data transmission between the device and the system, then pass the data collected in real time by the simulation platform to the plug-in module, and receive the control instructions output by it; S303: During the simulation process, the system status is monitored in real time through the HIL platform, dynamic events and system faults are detected, and the detected events and fault data are transmitted to the plug-in module during the detection, which triggers the control logic to generate corresponding control instructions, and transmits the instructions back to the HIL system; S304: using the adaptive module of the FPGA to automatically update the control parameters, and optimizing the parameters in real time based on the steepest descent algorithm; S305: The input, output and event logs during the simulation process are transmitted to the central monitoring system, and synchronized with the results of the HIL platform to record the status and control instructions of each API call.

5. The plug-in function and online testing method of a power system pre-training model according to claim 1, characterized in that: The steps to pre-train a reinforcement learning agent using an MPC controller in a HIL simulation to mimic the behavior of the MPC and gradually optimize the control strategy are: S401: Build an MPC controller model in the HIL simulation platform, and define the control time domain and prediction time domain based on the state space model of the system. Then, solve the optimal control instruction through the quadratic programming algorithm at each time step, and use its output as the benchmark control strategy; S402: running the MPC controller in the HIL platform, collecting system states and corresponding control instructions under different working conditions and dynamic events, recording state-action pair data, and storing them in a training data set as training data for the reinforcement learning agent; S403: Deploy the pre-trained reinforcement learning agent in the HIL simulation to receive system status and feedback data in real time. During the data reception process, the agent uses the Proximal Policy Optimization algorithm to continuously update the strategy and adaptively optimize its control behavior based on the real-time reward signal. S404: After each round of online training, evaluate the control strategy of the reinforcement learning agent and compare it with the baseline control strategy of the MPC, adjust the hyperparameters of the RL agent according to the evaluation results, and continue online training.

6. The plug-in function and online testing method of a power system pre-training model according to claim 5 is characterized in that: Build an MPC controller model in the HIL simulation platform, and define the control time domain and prediction time domain based on the state space model of the system. Then, solve the optimal control instruction through the quadratic programming algorithm at each time step, and use its output as the benchmark control strategy. The steps are as follows: Based on the operating characteristics of the power system, a state space model X(t+1)=AX(t)+BU(t)X(t+1) is established, where X(t) represents the system state, U(t) represents the control input, and A and B are the state and control matrices, respectively; Set the control time domain N c and prediction time domain N p , where N c Determine the number of steps, N, to apply the control command p Determine the prediction length of the future state of the system, and then, at each time step t, construct the objective function based on the current state X(t): Among them, Q and R are weight matrices, indicating the optimization priority of state and control input. Then, set the system constraint condition as U min ≤U(t)≤U max and status restrictions; At each time step, the optimal control input sequence U is solved by the quadratic programming algorithm. * ={U(t),U(t+1),...,U(t+N c -1)}, only the first control instruction U(t) in the sequence is applied to the system, and the rest is used for the next iterative update; Finally, the optimal control instruction U(t) after each round of solution is used as the benchmark control strategy, and its result is passed to the RL agent in the HIL simulation platform.

7. The plug-in function and online testing method of a power system pre-training model according to claim 6 is characterized in that: The steps to collect real-time data from HIL simulation and system operation, update model parameters and control strategies through optimization algorithms, and feed data back to RL agents and plug-in modules to adjust their behavior are: S501: Collecting real-time data from the HIL simulation platform and the system during operation and transmitting the collected data to a database through an API interface for storage; S502: Clean the collected real-time data, use a Kalman filter to smooth the noise data, detect and remove missing values ​​and abnormal data, and normalize the data into a format acceptable to the model; S503: Input the preprocessed data into the optimization algorithm, update the reinforcement learning agent model parameters through the particle swarm optimization algorithm, and adjust the control strategy in real time to adapt to the current system state.

8. The plug-in function and online testing method of a power system pre-training model according to claim 7, characterized in that: The steps to update the model parameters and adjust the control strategy in real time through the particle swarm optimization algorithm are: S5031: Initialize the particle swarm, and the position of each particle represents the model parameter X i =(x i1 , x i2 , dots, x id ), where i is the particle number and d is the parameter dimension. Then, initialize the velocity of each particle to V i =(v i1 , v i2 , dots, v id ), and obtain the optimal historical solution P of the particle individual i And the global optimal solution G, then define the fitness function f(X i ) Evaluate the performance of each particle; S5032: In each iteration, update the particle's velocity and position. The velocity update formula is V i (t+1)=ωV i (t)+c1r1(P i -X i (t))+c2r2(GX i (t)), where ω is the inertia weight, c1, c2 are learning factors, r1, r2 are random numbers (0, 1), and the position update formula is: X i (t+1)=X i (t)+V i (t+1), then, using the fitness function f(X i ) evaluates the new position of each particle. If f(X i (t+1))>f(P i ), then update the individual optimal solution P i =X i (t+1), if the fitness of a particle is better than the current global optimal solution G, then update the global optimal solution G = X i (t+1); S5033: The global optimal solution G is used as the latest parameter of the model, and the model is updated in real time through the API interface. The control strategy is generated according to the new parameters. The expression is: U(t) = K·X, where U(t) is the control instruction, K is the control matrix, and X(t) is the system state. Then, the feedback data is input into the particle swarm optimization algorithm, and the fitness f(X) is re-evaluated with the new state data. i ), and start the next round of optimization, repeating steps S5032 to S5035 to ensure that the reinforcement learning agent model continuously updates parameters and control strategies according to the real-time status.

Images