Plug-in function and online testing method for power system pre-trained model

Abstract

The application discloses a kind of plug-in function and online test method of power system pre-training model, and the present application is combined with NLP, BERT and reinforcement learning technology, and the pre-training model for electric power industry is designed, and the data-driven intelligent control strategy is realized. The generalization ability of the model is improved by integrating labeled and unlabeled data through semi-supervised learning, and a high-quality initialization model is provided for the reinforcement learning agent. The pre-trained model is disassembled into independent plug-in modules, integrated with the simulation platform through API, enabling on-demand loading and scenario switching of the modules, ensuring system flexibility. Simulate dynamic events and system failures on the HIL platform, pre-train the reinforcement learning agent with the MPC controller, and continuously adapt the strategy optimization process to the system state. In addition, the model parameters and control strategies are constantly updated through the particle swarm optimization algorithm to ensure that the RL agent and plug-in modules always synchronize with system requirements.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of plug-in testing, and particularly relates to a plug-in function and online testing method for a pre-trained model of a power system. BACKGROUND

[0002] With the expansion of the scale and the increase of the complexity of the power system, the traditional control strategy is difficult to adapt to the scheduling requirements under real-time changes and complex scenarios. Reinforcement learning (RL) as a data-driven control method can dynamically adapt to changes in system state, but its effectiveness depends on high-quality model parameters and policy optimization. At the same time, MPC controllers have been widely used in power systems due to their control capabilities in prediction and constraint conditions. However, how to effectively combine MPC and RL technologies and deeply integrate them with HIL simulation platforms to optimize control strategies in multiple scenarios is a key challenge in current smart power systems. In addition, real-time data collection and optimization, dynamic updating of model parameters, on-demand loading of plug-in modules, and closed-loop control capabilities of the simulation environment all pose higher requirements for the stability and responsiveness of the power system.

[0003] In view of this, the present application is proposed. SUMMARY

[0004] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art and provide a plug-in function and online testing method for a pre-trained model of a power system, which solves the problems raised in the background.

[0005] To solve the above technical problems, the basic idea of the technical solution of the present application is as follows:

[0006] A plug-in function and online testing method for a pre-trained model of a power system, comprising the following steps:

[0007] Step S1: using NLP, BERT and reinforcement learning technology, combining power industry data and standard documents, training a pre-trained model, and integrating labeled and unlabeled data through semi-supervised learning;

[0008] Step S2: split the pre-trained large model into independent plug-in modules, and integrate them with the simulation platform through API, and load them on demand according to different power scenarios;

[0009] Step S3: integrate the plug-in modules and physical devices on the HIL platform, and simulate dynamic events and system failures through real-time simulation;

[0010] Step S4: use the MPC controller to pre-train the reinforcement learning agent in the HIL simulation, so that it imitates the behavior of the MPC and gradually optimizes the control strategy;

[0011] Step S5: Collect real-time data from HIL simulation and system operation, and update reinforcement learning agent model parameters and control strategy through optimization algorithm, feed data back to RL agent and plug-in module to adjust its behavior.

[0012] Optionally, using NLP, BERT and reinforcement learning techniques, combined with power industry data and standard documents, pre-training models are trained, and steps for integrating labeled and unlabeled data through semi-supervised learning are as follows:

[0013] S101: Collect unlabeled data from operation logs, historical data and technical standards, obtain labeled data from maintenance records and fault reports, then clean, standardize and normalize the obtained data, and convert it into an input format acceptable by the BERT model;

[0014] S102: Input the preprocessed unlabeled data into the BERT model, randomly mask the vocabulary through the MLM task and predict the original content, and determine the logical relationship between texts through the NSP task, then use the Adam optimizer and execute distributed training through a GPU cluster to generate initial BERT model weights;

[0015] S103: Use the BERT model obtained through unsupervised pre-training to generate pseudo-labels for unlabeled data, and integrate the pseudo-labels with labeled data into an extended data set;

[0016] S104: Fine-tune the BERT model on the extended data set, optimize the prediction of labeled data through cross-entropy loss, and adjust the prediction results of unlabeled data using pseudo-label loss, and the fine-tuned BERT model is used as the initial model of the reinforcement learning agent.

[0017] Optionally, the steps for splitting the pre-trained large model into independent plug-in modules and integrating them into the simulation platform through API are as follows:

[0018] S201: According to different power scenarios, the functions of the pre-trained large model are disassembled into independent plug-in modules, then each plug-in module is mapped to a specific task, then each plug-in module is encapsulated using a microservice architecture, and an API interface is designed for it, defining the input and output formats;

[0019] S202: Configure the API address in the MATLAB / Simulink platform, and write a calling script to pass the input data of the simulation platform to the plug-in module and receive the prediction results or control instructions returned by the module;

[0020] S203: Dynamically load and switch plug-in modules according to different power scenarios and real-time data, and finally synchronize the running results of the plug-in modules with the data of the simulation platform, and use load balancing algorithms and multi-threading processing to ensure that there is no delay in module switching.

[0021] Optionally, the steps of integrating the plug-in module and the physical device on the HIL platform and simulating dynamic events and system failures through real-time simulation are as follows:

[0022] S301: The split plug-in module is communicated with the HIL platform through an API, and then the API address is configured to ensure that the plug-in module can receive data in real time and return control instructions;

[0023] S302: A preset simulation scene is built on the HIL platform, and then the physical device is connected to the HIL platform to synchronize data transmission between the device and the system, and then the data collected by the simulation platform in real time is transmitted to the plug-in module, and the control instructions output by the plug-in module are received;

[0024] S303: During the simulation process, the system state is monitored in real time through the HIL platform, dynamic events and system failures are detected, and the detected event and failure data are transmitted to the plug-in module to trigger the control logic to generate corresponding control instructions, and the instructions are transmitted back to the HIL system;

[0025] S304: The adaptive module of the FPGA is used to realize automatic updating of control parameters, and the parameters are optimized in real time based on the steepest descent algorithm;

[0026] S305: The input, output and event log in the simulation process are transmitted to the central monitoring system, and the results of the HIL platform are synchronized, and the state and control instructions of each API call are recorded.

[0027] Optionally, the steps of using an MPC controller to pre-train a reinforcement learning agent in HIL simulation so that the agent imitates the behavior of the MPC and gradually optimizes the control strategy are as follows:

[0028] S401: An MPC controller model is built in the HIL simulation platform, and the control time domain and the prediction time domain are defined based on the state space model of the system, and then the optimal control instruction is solved at each time step through a quadratic programming algorithm, and the output is used as the benchmark control strategy;

[0029] S402: The MPC controller is run in the HIL platform, the system state and the corresponding control instruction are collected under different working conditions and dynamic events, the state-action pair data are recorded, and stored in a training data set as training data for the reinforcement learning agent;

[0030] S403: The pre-trained reinforcement learning agent is deployed in HIL simulation, and the system state and feedback data are received in real time. During the data receiving process, the agent uses the Proximal Policy Optimization algorithm to continuously update the strategy, and optimizes its control behavior based on the real-time reward signal;

[0031] S404: After each round of online training, evaluate the control policy of the reinforcement learning agent, compare it with the benchmark control policy of the MPC, adjust the hyperparameters of the RL agent according to the evaluation results, and continue online training.

[0032] Optionally, build an MPC controller model in the HIL simulation platform, define the control horizon and prediction horizon based on the state space model of the system, then solve the optimal control instruction at each time step by a quadratic programming algorithm, and output it as the benchmark control policy step:

[0033] Based on the operating characteristics of the power system, a state space model X(t+1) = AX(t) + BU(t)X(t+1) is established, where X(t) represents the system state, U(t) represents the control input, and A and B are state and control matrices, respectively;

[0034] Set the control horizon N c and the prediction horizon N p , where N c determines the number of steps for which the control instruction is applied, and N p decides the prediction length of the future state of the system, then at each time step t, based on the current state X(t), the objective function is constructed: where Q and R are weight matrices representing the optimization priority of state and control input, then set the system constraint condition as U min ≤ U(t) ≤ U max and state limits;

[0035] At each time step, the optimal control input sequence U * ={U(t), U(t+1),..., U(t+N c -1)} is solved by a quadratic programming algorithm, only the first control instruction U(t) in the sequence is applied to the system, and the rest is used for next iteration update;

[0036] Finally, the optimal control instruction U(t) solved at each round is taken as the benchmark control policy, and its result is passed to the RL agent in the HIL simulation platform.

[0037] Optionally, collect real-time data from HIL simulation and system operation, and update model parameters and control strategies through optimization algorithms, and feed the data back to the RL agent and plug-in module to adjust its behavior step:

[0038] S501: Collect real-time data from the HIL simulation platform and the system operation process and transmit the collected data to the database through the API interface for storage;

[0039] S502: Clean the collected real-time data, smooth the noise data using Kalman filter, detect and remove missing values and abnormal data, and normalize the data into a format acceptable to the model;

[0040] S503: Input the preprocessed data into the optimization algorithm, update the reinforcement learning agent model parameters through the particle swarm optimization algorithm, and adjust the control strategy in real time to adapt to the current system state.

[0041] Optionally, the step of updating the model parameters and adjusting the control strategy in real time through the particle swarm optimization algorithm is:

[0042] S5031: Initialize the particle swarm, and the position of each particle represents the model parameters X i =(x i1 , x i2 , dots, x id ), where i is the particle number, d is the parameter dimension, then initialize the speed of each particle as V i =(v i1 , v i2 , dots, v id ), and obtain the individual historical optimal solution P i and the global optimal solution G, then define the fitness function f(X i ) to evaluate the performance of each particle;

[0043] S5032: In each iteration, update the speed and position of the particle, the speed update formula is V i (t+1) = ωV i (t) + c1r1(P i -X i (t)) + c2r2(G-X i (t)), where ω is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers (0, 1), the position update formula is X i (t+1) = X i (t) + V i (t+1), then use the fitness function f(X i ) to evaluate the new position of each particle, if f(X i (t+1)) > f(P i ), update the individual optimal solution P i = X i (t+1). If the fitness of a particle is better than the current global optimal solution G, update the global optimal solution G = X i (t+1);

[0044] S5033: Take the obtained global optimal solution G as the latest parameters of the model, update the model in real time through the API interface, and generate a control strategy according to the new parameters, the expression of which is: U(t) = K X, where U(t) is a control instruction, K is a control matrix, and X(t) is a system state, then, input the feedback data into the particle swarm optimization algorithm, reevaluate the fitness f(X i ), and start the next round of optimization, repeat steps S5032 to S5035, and ensure that the reinforcement learning agent model continuously updates the parameters and control strategy according to the real-time state.

[0045] After the above technical solutions are adopted, the present application has the following beneficial effects compared with the prior art, of course, any product implementing the present application does not necessarily need to achieve all the advantages described below:

[0046] The present application designs a pre-training model for the power industry by combining NLP, BERT and reinforcement learning technology, and realizes a data-driven intelligent control strategy. Through semi-supervised learning, the model's generalization ability is improved, and a high-quality initialization model is provided for the reinforcement learning agent. The pre-training model is disassembled into independent plug-in modules, integrated with the simulation platform through API, and the on-demand loading and scene switching of the modules are realized, ensuring the flexibility of the system. Dynamic events and system failures are simulated on the HIL platform, and the reinforcement learning agent is pre-trained in combination with the MPC controller, so that its strategy optimization process can continuously adapt to the system state. In addition, the model parameters and control strategy are continuously updated through the particle swarm optimization algorithm, ensuring that the RL agent and plug-in modules are always synchronized with the system requirements.

[0047] The specific embodiments of the present application will be described in further detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0048] The drawings in the following description are only some embodiments, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings. In the drawings

[0049] In the drawings:

[0050] Figure 1 Plug-in function and online test method flowchart for power system pre-training model.

[0051] It should be noted that these drawings and written descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0052] The present application will now be described in further detail with reference to the accompanying drawings.

[0053] Referring to Figure 1 As shown in the embodiment, a plug-in function and an online testing method of a power system pre-training model are provided, including the following steps:

[0054] Step S1: using NLP, BERT and reinforcement learning technology, combining power industry data and standard documents, training pre-training model, and integrating labeled and unlabeled data through semi-supervised learning;

[0055] Step S2: split the pre-training large model into independent plug-in modules, and integrate with the simulation platform through API, and load according to different power scenarios on demand;

[0056] Step S3: integrate plug-in modules and physical devices on the HIL platform, simulate dynamic events and system failures through real-time simulation;

[0057] Step S4: use MPC controller to pre-train reinforcement learning agent in HIL simulation, so as to imitate the behavior of MPC and gradually optimize the control strategy;

[0058] Step S5: collect real-time data from HIL simulation and system operation, and update reinforcement learning agent model parameters and control strategy through optimization algorithm, and feed back the data to RL agent and plug-in module to adjust its behavior.

[0059] In the embodiment, using NLP, BERT and reinforcement learning technology, combining power industry data and standard documents, training pre-training model, and integrating labeled and unlabeled data through semi-supervised learning are as follows:

[0060] S101: collect unlabeled data from operation logs, historical data and technical standards, obtain labeled data from maintenance records and fault reports, then clean, standardize and normalize the obtained data, and convert it into the input format acceptable by the BERT model; unlabeled data comes from daily operation data, real-time monitoring logs and technical standard documents of the power system, which do not carry specific labels such as device status or operation instructions. Labeled data comes from device fault reports, maintenance records and manually annotated data sets, containing explicit label information;

[0061] S102: input the preprocessed unlabeled data into the BERT model, randomly mask the vocabulary through the MLM task and predict the original content, and determine the logical relationship between texts through the NSP task, then use the Adam optimizer and execute distributed training through GPU cluster to generate initial BERT model weights;

[0062] S103: Use the BERT model obtained by unsupervised pre-training to generate pseudo labels for unlabeled data, and integrate the pseudo labels with labeled data into an extended data set;

[0063] S104: Fine-tune the BERT model on the extended data set, optimize the prediction of labeled data through cross-entropy loss, and adjust the prediction result of unlabeled data using pseudo label loss. The fine-tuned BERT model is used as the initial model of the reinforcement learning agent.

[0064] The step of the embodiment of splitting the pre-trained large model into independent plug-in modules and integrating through the API simulation platform is:

[0065] S201: According to different power scenarios (such as frequency control, load management, and equipment maintenance), the functions of the pre-trained large model are disassembled into independent plug-in modules. Then, each plug-in module is mapped to a specific task. Subsequently, each plug-in module is encapsulated using a microservice architecture, and an API interface is designed for it, and the input and output formats (such as JSON or XML) are defined;

[0066] S202: Configure the API address in the MATLAB / Simulink platform, and write a calling script to pass the input data of the simulation platform to the plug-in module and receive the prediction results or control instructions returned by the module.

[0067] S203: According to different power scenarios and real-time data, dynamically load and switch plug-in modules. Finally, synchronize the running results of the plug-in modules with the data of the simulation platform, and use load balancing algorithms and multi-threading processing to ensure that there is no delay in module switching.

[0068] The step of the embodiment of integrating plug-in modules and physical devices on the HIL platform and simulating dynamic events and system failures through real-time simulation is:

[0069] S301: The split plug-in modules are communicated with the HIL platform through API, and then the API address is configured to ensure that the plug-in modules can receive data in real time and return control instructions.

[0070] S302: Build a preset simulation scenario on the HIL platform, including normal operation state, load change and system failure. Then, connect the physical devices to the HIL platform to synchronize data transmission between the devices and the system. Subsequently, pass the data collected by the simulation platform in real time to the plug-in module, and receive the output control instructions.

[0071] S303: During the simulation process, the system state is monitored in real time through the HIL platform to detect dynamic events and system failures. When detecting, pass the detected event and fault data to the plug-in module to trigger the control logic to generate corresponding control instructions, and pass the instructions back to the HIL system;

[0072] S304: The adaptive module of the FPGA is used to realize automatic updating of the control parameters, and the parameters are optimized in real time based on the steepest descent algorithm;

[0073] S305: The input, output and event log in the simulation process are transmitted to the central monitoring system, and the results of the HIL platform are synchronized, and the state and control instruction of each API call are recorded.

[0074] In the embodiment, the steps of using the MPC controller to pre-train the reinforcement learning agent in the HIL simulation, so that the reinforcement learning agent imitates the behavior of the MPC and gradually optimizes the control strategy are as follows:

[0075] S401: Build an MPC controller model in the HIL simulation platform, define the control time domain and the prediction time domain based on the state space model of the system, then solve the optimal control instruction at each time step by the quadratic programming algorithm, and output it as the benchmark control strategy;

[0076] S402: Run the MPC controller in the HIL platform, collect the system state and the corresponding control instruction under different working conditions and dynamic events, record the state-action pair data, and store them in the training data set as the training data of the reinforcement learning agent;

[0077] S403: Deploy the pre-trained reinforcement learning agent in the HIL simulation, and receive the system state and feedback data in real time. The agent continuously updates the strategy using the Proximal Policy Optimization algorithm during the data receiving process, and optimizes its control behavior based on the real-time reward signal;

[0078] S404: After each round of online training, evaluate the control strategy of the reinforcement learning agent, compare it with the benchmark control strategy of the MPC, adjust the hyperparameters of the RL agent according to the evaluation results, and continue online training.

[0079] In the embodiment, the steps of building an MPC controller model in the HIL simulation platform, defining the control time domain and the prediction time domain based on the state space model of the system, and then solving the optimal control instruction at each time step by the quadratic programming algorithm to output it as the benchmark control strategy are as follows:

[0080] Based on the operating characteristics of the power system, a state space model X(t+1) = AX(t) + BU(t)X(t+1) is established, where X(t) represents the system state, U(t) represents the control input, and A and B are state and control matrices, respectively;

[0081] Set the control time domain N c and the prediction time domain N p , where N cDetermine the number of steps N for applying the control command. p Determine the prediction length of the system's future states. Then, at each time step t, construct the objective function based on the current state X(t): Where Q and R are weight matrices, representing the optimization priorities of the state and control inputs, and then the system constraints are set as U. min ≤U(t)≤U max and status restrictions;

[0082] The optimal control input sequence U is solved using a quadratic programming algorithm at each time step. * ={U(t),U(t+1),...,U(t+N c -1)}, only the first control instruction U(t) in the sequence is applied to the system, and the rest is used for the next iteration update;

[0083] Finally, the optimal control command U(t) obtained after each round of solving is used as the baseline control strategy, and its result is passed to the RL agent in the HIL simulation platform.

[0084] This embodiment achieves precise control of the power system by building an MPC controller model on the HIL simulation platform and defining the control time domain and prediction time domain based on the system's state-space model. A quadratic programming algorithm is used to solve for the optimal control input sequence at each time step, ensuring efficient system operation under constraints. By setting weight matrices for the state and control inputs, priority coordination among different control objectives is ensured. Furthermore, only the first control instruction of the current time step is applied, with subsequent instructions deferred to the next iteration, enhancing the flexibility and real-time performance of the control. The MPC control strategy serves as a benchmark, providing a stable initial strategy for the reinforcement learning agent, enabling the RL agent to progressively optimize control behavior within the simulation platform, ensuring the system's responsiveness and robustness in dynamic environments. This control method, based on the combination of MPC and RL, achieves an organic fusion of precise prediction and optimal control, improving the stability and adaptability of the power system in complex scenarios.

[0085] In this embodiment, the steps of collecting real-time data from HIL simulation and system operation, updating model parameters and control strategies through optimization algorithms, and feeding the data back to the RL agent and plug-in modules to adjust their behavior are as follows:

[0086] S501: Collects real-time data from the HIL simulation platform and system operation process, and transmits the collected data to the database for storage via API interface;

[0087] S502: Clean the collected real-time data, use a Kalman filter to smooth noisy data, detect and remove missing values ​​and outliers, and normalize the data to a format acceptable to the model.

[0088] S503: input the preprocessed data into the optimization algorithm, update the reinforcement learning agent model parameters through the particle swarm optimization algorithm, and adjust the control strategy in real time to adapt to the current system state.

[0089] In this embodiment, the steps of updating the model parameters and adjusting the control strategy in real time through the particle swarm optimization algorithm are as follows:

[0090] S5031: initialize the particle swarm, and the position of each particle represents the model parameters X i =(x i1 , x i2 , dots, x id ), where i is the particle number and d is the parameter dimension, then initialize the speed of each particle as V i =(v i1 , v i2 , dots, v id ), and obtain the individual historical optimal solution P i and the global optimal solution G, then define the fitness function f(X i ) to evaluate the performance of each particle;

[0091] S5032: in each iteration, update the speed and position of the particle, the speed update formula is V i (t+1) = ωV i (t) + c1r1(P i -X i (t)) + c2r2(G-X i (t)), where ω is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers (0, 1), the position update formula is X i (t+1) = X i (t) + V i (t+1), then use the fitness function f(X i ) to evaluate the new position of each particle, if f(X i (t+1)) > f(P i ), update the individual optimal solution P i = Xx(t+1). If the fitness of a particle is better than the current global optimal solution G, update the global optimal solution G = X i (t+1);

[0092] S5033: Take the obtained global optimal solution G as the latest parameters of the model, update the model in real time through the API interface, and generate the control strategy according to the new parameters, the expression of which is: U(t) = K X, where U(t) is the control instruction, K is the control matrix, and X(t) is the system state. Then, input the feedback data into the particle swarm optimization algorithm, re-evaluate the fitness f(X i ) with the new state data, and start the next round of optimization. Repeat steps S5032 to S5035 to ensure that the reinforcement learning agent model continuously updates the parameters and control strategy according to the real-time state.

[0093] It should be noted that the method of updating model parameters and adjusting control strategy in real time through the particle swarm optimization algorithm ensures that the model can dynamically adapt to the real-time state changes of the power system and guarantees the continuous optimization of the control strategy. By initializing the particle swarm, defining the fitness function, and iteratively updating the position and speed of the particles, the global optimal solution is found in real time. In each iteration, the global optimal solution is used as the latest parameter of the model using the quadratic optimization result, and the parameter is applied to the system control strategy in real time through the API interface, ensuring that the strategy can follow the system state changes for adjustment. In addition, by inputting the feedback data into the optimization algorithm in a loop, the reinforcement learning agent can continuously update the model parameters and control strategy, avoiding system rigidity and ensuring its stability and accuracy in multiple scenarios. This dynamic adaptive optimization mechanism improves the decision-making efficiency of the model in complex power scenarios and reduces control delay.

[0094] Embodiment 1: In this embodiment, a pre-trained model for power systems is constructed based on NLP, BERT, and reinforcement learning techniques, and is split into plug-in modules, which are tested for performance on a HIL simulation platform. The experiment verifies the functions of each module in different power scenarios, and uses an MPC controller as the initial strategy of the reinforcement learning agent for optimization.

[0095] The BERT model is pre-trained unsupervised on a GPU cluster, using 50GB of data collected from power system historical operation logs, monitoring data, and fault reports, of which 10% is labeled data. The Adam optimizer (learning rate 5e-5) is used for training for 48 hours. After training, pseudo labels are generated for unlabeled data, which are integrated with labeled data to form an expanded dataset, with a data volume of 70GB. Fine-tuning is performed to form the initial model of the reinforcement learning agent.

[0096] The pre-trained model is split into five plug-in modules for prediction, alarm, control, policy management, and data synchronization. Each module is integrated into the MATLAB / Simulink simulation platform through RESTful API, and five power scenarios such as load variation and line short circuit are built on the platform. The MPC controller in the HIL platform defines the prediction time domain (20 steps) and the control time domain (5 steps), and collects voltage, current and frequency data under different working conditions to generate 5000 state-action pair data for the initial training data of the RL agent.

[0097] The reinforcement learning agent is run on the HIL platform, trained online for 10 hours using the PPO algorithm, and realizes real-time feedback through the API interface. The particle swarm optimization algorithm (PSO) is used to optimize the model parameters of the RL agent, and the control strategy is updated every second to adjust the agent's behavior in real time according to the changes in system state, ensuring the adaptability of policy optimization and system response.

[0098] Experiments show that the response speed of the control strategy optimized by reinforcement learning is significantly improved compared to before optimization. In dynamic scenarios such as load mutation and equipment failure, the response time is reduced by more than 40% on average, and the control error is reduced from 5%-8% of the MPC strategy to 2%-3% after RL optimization. For example, in the load mutation scenario, the optimized response time is reduced to 2.1 seconds, which is more advantageous than the 3.5 seconds of the MPC. In addition, in the line short circuit scenario, the control error of the RL agent is reduced from 7.1% of the MPC strategy to 3.5%. The system realizes non-delay switching and efficient operation of plug-in modules through multi-threading and load balancing, ensuring data synchronization and real-time policy update during simulation. The overall results show that this method performs well in complex power system scenarios in terms of stability and adaptability.

[0099] The present application is not limited to the above-mentioned embodiments, and any person should know that structural changes made under the inspiration of the present application fall within the scope of the present application. Any technical, shape, or structure not described in detail in the present application is a known technology.

Claims

1. A plug-in function and online testing method for a pre-trained power system model, characterized in that, Includes the following steps: Step S1: Using NLP, BERT and reinforcement learning techniques, combined with power industry data and standard documents, train a pre-trained model, and integrate labeled and unlabeled data through semi-supervised learning; Step S2: Decompose the pre-trained large model into independent plug-in modules and integrate them with the simulation platform through API, loading them as needed according to different power scenarios; Step S3: Integrate plug-in modules and physical devices on the HIL platform, and simulate dynamic events and system failures through real-time simulation: S301: Establish communication between the split plug-in modules and the HIL platform via API. Then, configure the API address to ensure that the plug-in modules can receive data and return control commands in real time. S302: Build a preset simulation scenario on the HIL platform. Then, connect the physical device to the HIL platform to synchronize the data transmission between the device and the system. Subsequently, transmit the data collected in real time by the simulation platform to the plug-in module and receive the control commands output by it. S303: During the simulation, the system status is monitored in real time through the HIL platform to detect dynamic events and system faults. During the detection, the detected event and fault data are transmitted to the plug-in module, triggering the control logic to generate corresponding control commands, and the commands are transmitted back to the HIL system. S304: The adaptive module of the FPGA is used to realize the automatic updating of control parameters and optimize the parameters in real time based on the steepest descent algorithm; S305: Transmit the input, output, and event logs during the simulation process to the central monitoring system and synchronize them with the results of the HIL platform, recording the status and control commands of each API call; Step S4: Use the MPC controller to pre-train a reinforcement learning agent in the HIL simulation, making it mimic the behavior of the MPC and gradually optimize the control strategy: S401: Build an MPC controller model in the HIL simulation platform, and define the control time domain and prediction time domain based on the system's state space model. Then, solve for the optimal control command at each time step using a quadratic programming algorithm, and use its output as the baseline control strategy. S402: Runs the MPC controller in the HIL platform, collects system status and corresponding control commands under different operating conditions and dynamic events, records state-action pair data, and stores it in the training dataset as training data for reinforcement learning agents; S403: Deploy the pre-trained reinforcement learning agent in the HIL simulation to receive system status and feedback data in real time. During the data reception process, the agent continuously updates its policy using the Proximal Policy Optimization algorithm and adaptively optimizes its control behavior based on the real-time reward signal. S404: After each round of online training, evaluate the control strategy of the reinforcement learning agent, compare it with the baseline control strategy of MPC, adjust the hyperparameters of the RL agent according to the evaluation results, and continue online training; Step S5: Collect real-time data from HIL simulation and system operation, update the parameters of the reinforcement learning agent model and the control strategy through optimization algorithms, and feed the data back to the RL agent and plug-in modules to adjust their behavior.

2. The plug-in function and online testing method for a power system pre-trained model according to claim 1, characterized in that, The steps involved in training a pre-trained model using NLP, BERT, and reinforcement learning techniques, combined with power industry data and standard documents, and then integrating labeled and unlabeled data through semi-supervised learning are as follows: S101: Collect unlabeled data from operation logs, historical data and technical standards, obtain labeled data from maintenance records and fault reports, and then clean, standardize and normalize the obtained data to transform it into an input format acceptable to the BERT model. S102: Input the preprocessed unlabeled data into the BERT model, randomly mask words and predict the original content through the MLM task, and determine the logical relationship between texts through the NSP task. Then, use the Adam optimizer and perform distributed training through a GPU cluster to generate the initial BERT model weights. S103: Use the BERT model obtained from unsupervised pre-training to generate pseudo-labels for unlabeled data, and integrate the pseudo-labels with labeled data to form an extended dataset; S104: Fine-tune the BERT model on the extended dataset. Optimize the predictions for labeled data using cross-entropy loss and adjust the predictions for unlabeled data using pseudo-label loss. The fine-tuned BERT model serves as the initial model for the reinforcement learning agent.

3. The plug-in function and online testing method for a power system pre-trained model according to claim 1, characterized in that, The steps to break down a pre-trained large model into independent plug-in modules and integrate them into a simulation platform via API are as follows: S201: Based on different power scenarios, the functions of the pre-trained large model are decomposed into independent plug-in modules. Then, each plug-in module is mapped to a specific task. Subsequently, each plug-in module is encapsulated using a microservice architecture, and API interfaces are designed for it, defining input and output formats. S202: Configure the API address in the simulation platform and write the calling script to pass the input data of the simulation platform to the plug-in module and receive the prediction results or control instructions returned by the module. S203: Dynamically load and switch plug-in modules according to different power scenarios and real-time data. Finally, synchronize the running results of the plug-in modules with the data of the simulation platform, and use load balancing algorithms and multi-threaded processing to ensure that there is no delay in module switching.

4. The plug-in function and online testing method for a power system pre-trained model according to claim 1, characterized in that, The steps for building an MPC controller model in the HIL simulation platform, defining the control time domain and prediction time domain based on the system's state-space model, solving for the optimal control command at each time step using a quadratic programming algorithm, and using the output as the baseline control strategy are as follows: Based on the operating characteristics of the power system, a state-space model is established. ,in, Indicates the system status. Indicates control input, and These are the state and control matrices, respectively. Set control time domain and prediction time domain ,in Determine the number of steps for applying control commands. Determine the prediction length for the future state of the system, and then, at each time step... Based on the current state Construct the objective function: ,in, and The weight matrix represents the optimization priority of the state and control inputs. Next, the system constraints are set as follows: and status restrictions; The optimal control input sequence is solved using a quadratic programming algorithm at each time step. Only the first control instruction in the sequence is applied. The remainder is used for the next iteration update; Finally, the optimal control command after each round of solving will be... It serves as a baseline control strategy, and its results are passed to the RL agent in the HIL simulation platform.

5. The plug-in function and online testing method for a power system pre-trained model according to claim 4, characterized in that, The steps for collecting real-time data from HIL simulations and system operation, updating model parameters and control strategies through optimization algorithms, and feeding the data back to RL agents and plug-in modules to adjust their behavior are as follows: S501: Collects real-time data from the HIL simulation platform and system operation process, and transmits the collected data to the database for storage via API interface; S502: Clean the collected real-time data, use a Kalman filter to smooth noisy data, detect and remove missing values ​​and outliers, and normalize the data to a format acceptable to the model. S503: Input the preprocessed data into the optimization algorithm, update the reinforcement learning agent model parameters through the particle swarm optimization algorithm, and adjust the control strategy in real time to adapt to the current system state.

Images