A hybrid multi-objective coordinated control system based on SVG and synchronous condenser

Abstract

The application relates to the technical field of power systems, and discloses a hybrid multi-target coordinated control system based on SVG and synchronous phase modulators. By fusing real-time states and future predictions, the decision quality and foresight of the intelligent agent are significantly improved. Compared with a single system state vector, the enhanced state representation not only contains real-time parameters such as voltage deviation and reactive power difference, but also integrates future uncertainty probability information predicted by a random forest, so that the intelligent agent can simultaneously perceive the current state and expected evolution trend of the system. The enhanced environmental perception capability enables the Q learning algorithm to make more predictive decisions. The coordination strategy of the SVG and the phase modulator is adjusted in advance before voltage fluctuation occurs. The enhanced state representation provides comprehensive environmental information, so that the intelligent agent can better balance the relationship between fast response and steady-state support, guarantee voltage stability, improve the economy of system operation, and finally realize true multi-target collaborative optimization.

Description

Technical Field

[0001] This invention relates to the field of power system technology, and in particular to a hybrid multi-objective coordinated control system based on SVG and synchronous condenser. Background Technology

[0002] Existing technologies typically employ traditional control methods to coordinate SVG (Static Var Generator) and synchronous condenser. Specifically, this is mainly achieved through PID control with preset parameters or fuzzy control based on a rule base. The PID controller generates control signals based on the deviations of controlled variables such as voltage and reactive power from the target values, using proportional, integral, and derivative operations, and sends these signals to both the SVG and the synchronous condenser. Fuzzy control, on the other hand, uses fuzzy rules set by expert experience to convert input quantities such as system deviations and rates of change of deviation into fuzzy quantities. After rule-based reasoning and defuzzification, it outputs relatively fixed coordinated control commands. The entire control process relies on pre-tuned parameters or rules, and the system responds to changes in the power grid according to fixed logic.

[0003] The aforementioned and existing related technologies often suffer from the following drawbacks: the control parameters or rule bases of traditional methods are fixed and cannot adapt to the high uncertainty brought about by sudden load changes and power fluctuations of new energy sources in the power grid. This results in a control blind zone between the fast response of the SVG and the slow stabilization of the synchronous condenser, leading to a dynamic mismatch between the two. Consequently, problems such as excessive voltage flicker and low coordination efficiency arise, making it impossible to achieve optimal coordination. Summary of the Invention

[0004] The technical problem to be solved by this invention is that the existing technology has the disadvantage of causing a control blind zone between the fast response of SVG and the slow stability of synchronous condenser, resulting in a dynamic mismatch between the two. To this end, we propose a hybrid multi-objective coordinated control system based on SVG and synchronous condenser.

[0005] To achieve the above objectives, this application adopts the following technical solution: a hybrid multi-objective coordinated control system based on SVG and synchronous condenser, comprising:

[0006] The real-time status monitoring module is used to continuously collect power grid operating parameters through a multi-sensor network and construct the system state space using the power grid operating parameters based on Markov decision process theory.

[0007] The probabilistic decision support module collects historical and real-time system state space data, performs uncertainty prediction and probabilistic representation based on the random forest algorithm, and finally outputs the system uncertainty prediction results to the reinforcement learning decision module.

[0008] The reinforcement learning decision-making module uses the Q-learning algorithm to build a reward and punishment model, generates reactive power compensation strategies through dynamic exploration and utilization mechanisms, and outputs coordinated control commands for SVG and synchronous condenser.

[0009] The collaborative execution control module is used to receive coordinated control commands, parse and distribute action signals to the controllers of the SVG and the synchronous condenser, so as to achieve dynamic coordination of fast response and steady-state compensation and complete multi-objective optimized control.

[0010] Furthermore, the system continuously collects power grid operating parameters through a multi-sensor network and constructs a system state space based on a Markov decision process, then performs the following operations:

[0011] By utilizing voltage transformers, current transformers, and phase measurement equipment deployed at key nodes of the power grid, the original power grid operating parameters, including voltage, current, phase, reactive power, and frequency, are acquired in real time at a millisecond-level sampling frequency. The power grid operating parameters are then filtered and outlier removed.

[0012] The preprocessed power grid operating parameters are mapped into multi-dimensional state variables including voltage deviation, reactive power difference, response delay, and load fluctuation trend to construct a system state vector.

[0013] Based on Markov decision processes, the system state vector transition process is modeled as a discrete-time decision problem. Each state contains historical operating information and current state characteristics, enabling the state space to represent the dynamic evolution of the system and providing a temporally correlated state representation for reinforcement learning decision modules.

[0014] The state representation is transmitted to the reinforcement learning decision module via a protocol conversion gateway.

[0015] Furthermore, the system collects historical and real-time state space data, performs uncertainty prediction and probabilistic representation based on the random forest algorithm, and finally outputs the system uncertainty prediction results to the reinforcement learning decision module to perform the following operations:

[0016] Historical and real-time system status data are collected, and load change rate, new energy output fluctuation trend and external environmental characteristics are extracted from them as training and prediction inputs to construct a sample set for the random forest algorithm.

[0017] The Bootstrap sampling method is used to generate multiple sub-training sets from the sample set, and a decision tree is constructed for each sub-training set. During the decision tree generation process, some features are randomly selected for node splitting.

[0018] Using the system state vector from the previous moment as input, parallel reasoning is performed through multiple decision trees in a trained random forest model, with each tree outputting a prediction of the direction of load or renewable energy power fluctuations in the short term.

[0019] The prediction results of all decision trees are statistically analyzed, and the final fluctuation direction classification is determined by voting mechanism or the probability of continuous fluctuation is calculated by mean, generating a probabilistic representation of system uncertainty.

[0020] The probabilistic representation results are used as auxiliary state variables, and the auxiliary state variables are concatenated with the system state vector to form an enhanced state representation, which is used in the decision-making process of the Q-learning algorithm.

[0021] Furthermore, the Q-learning algorithm is used to construct a reward and punishment model, and a reactive power compensation strategy is generated through a dynamic exploration and utilization mechanism to perform the following operations:

[0022] Based on the state representation of the input, a reward function with the core elements of minimizing voltage deviation, minimizing reactive power difference and minimizing system loss is used. The reward function constructs a multi-objective optimization evaluation system through weighted summation to quantitatively evaluate the merits of the state actions selected by the built-in intelligent agent of the reinforcement learning decision module from the predefined action space.

[0023] The agent then adopts an ε-greedy strategy to balance the exploration and utilization process. By fitting the state-action value function through the Q-table and continuously updating and optimizing it, the agent is finally guided to autonomously learn a collaborative compensation strategy that takes into account both dynamic performance and static economy, so as to achieve the organic cooperation between SVG and synchronous condenser.

[0024] Furthermore, the reward function constructs a multi-objective optimization evaluation system through weighted summation, and performs the following operations:

[0025] The mathematical expression for the reward function is defined as follows: in: Indicates time step The overall reward value, For the weighting coefficients, satisfying , For actual voltage measurement, The system's rated voltage. This represents the actual reactive power of the system. For the current target reactive power demand, To compensate for the total active power loss generated by the SVG and synchronous condenser during the process;

[0026] By adjusting the weighting coefficients The numerical value, dynamically configuring the preference direction of the multi-objective optimization evaluation system;

[0027] The real-time collected power grid operating parameters are substituted into the reward function calculation formula to solve for the comprehensive reward value at the current time step. ;Will As an immediate reward input to the Q-learning algorithm, it is used to update the state-action value function, driving the agent to learn towards the optimal multi-objective policy.

[0028] Furthermore, the agent then employs an ε-greedy strategy to balance the exploration and utilization process, fitting the state-action value function using a Q-table and continuously updating and optimizing it. Ultimately, the agent is guided to autonomously learn a collaborative compensation strategy that balances dynamic performance and static economy, and performs the following operations:

[0029] Set an initial value for the exploration rate ε and a decay strategy. Generate a random number at each decision time. When the random number is less than the current exploration rate ε, the agent randomly selects an action from the action space to explore the unknown strategy space. Otherwise, it selects the optimal action with the highest value according to the current Q table.

[0030] After performing the selected action, observe the next state and immediate reward from the environment. Calculate new state action values ​​based on the Q-learning update rules;

[0031] The corresponding entries in the Q table are updated with new state-action values ​​to gradually optimize the estimation of state-action value. At the same time, the exploration rate ε is decayed according to a preset period to gradually reduce the probability of random exploration and increase the utilization of the learned optimal strategy.

[0032] The above process is iterated until the Q-table converges and meets the predetermined requirements, so that the agent can finally obtain a stable optimal policy and adaptively output the best coordinated control command of SVG and synchronous modulator based on the real-time state representation.

[0033] Furthermore, after executing the selected action, the next state and immediate reward from the observed environment are used to calculate a new state-action value according to the Q-learning update rule, and the following operations are performed:

[0034] After performing the selected action, the agent first observes the next state fed back by the environment. Compared with the immediate reward calculated by the reward function Then, based on the Q-learning update rules, actions are performed according to the current state. Q value Instant rewards and the next state The maximum expected Q value of all possible actions Through calculation formula To obtain new state action values, where The learning rate is used to control the update step size. The discount factor is calculated using time-series difference error. This is used to gradually correct the estimation bias of the state action values ​​in the Q table, thereby achieving iterative optimization of the action value function.

[0035] Furthermore, the step of concatenating the auxiliary state variables with the system state vector to form an enhanced state representation involves the following operations:

[0036] It receives the system uncertainty prediction results from the probabilistic decision support module as auxiliary state variables, and at the same time receives the original system state vector from the real-time state monitoring module;

[0037] The auxiliary state variables and the system state vector are preprocessed to align their dimensions, ensuring that the numerical range of the two vectors matches their dimensions.

[0038] An auxiliary state variable is appended to the end of the system state vector using a vector concatenation algorithm to generate an enhanced state representation;

[0039] The spliced ​​enhanced state representation is standardized to eliminate numerical differences caused by different physical dimensions, ensuring the convergence stability of the subsequent Q-learning algorithm.

[0040] The processed enhanced state representation is fed into the state buffer of the reinforcement learning decision module in real time, providing the Q-learning algorithm with complete environmental information including the current system state space and future uncertainty predictions.

[0041] Furthermore, the mathematical expression for the vector concatenation algorithm is: ,in, This represents an enhanced state representation. Represents the original system state vector. Represents auxiliary state variables, This indicates a vector concatenation operation.

[0042] Furthermore, the processed enhanced state representation is fed into the state buffer of the reinforcement learning decision module in real time, providing the Q-learning algorithm with complete environmental information including the current system state and future uncertainty predictions, and performing the following operations:

[0043] Establish a state buffer data interface protocol and define the data format and transmission frequency for enhanced state representation;

[0044] The enhanced state representation is written into a circular queue of the state buffer through memory mapping, and the producer-consumer model is used to realize real-time data updating and reading.

[0045] Add timestamps and sequence numbers to each augmented state representation in the state buffer to ensure that the Q-learning algorithm processes state data in the correct time sequence;

[0046] Based on the state verification mechanism, the integrity of the enhanced state representation stored in the buffer is verified, and abnormal or timed-out data is discarded.

[0047] The enhanced state representation processed in the buffer is provided to the Q-learning algorithm on demand through the API interface, enabling it to simultaneously obtain the preprocessed current system state space and the future uncertainty probability information predicted by the random forest.

[0048] The technical effects and advantages of this invention are as follows:

[0049] In this invention, a precise system state space is constructed through a real-time state monitoring module, providing millisecond-level data support for decision-making; a reinforcement learning decision module autonomously generates coordination strategies based on the Q-learning algorithm, dynamically optimizing reactive power compensation instructions, completely overcoming the shortcomings of traditional PID control parameters being fixed and fuzzy control rules being limited; a collaborative execution control module accurately parses instructions and distributes them for execution, ensuring the organic coordination between SVG fast response and synchronous condenser steady-state compensation. This system stably controls voltage deviation within ±1%, reduces response delay by 30%, and significantly improves grid voltage stability and power quality.

[0050] In this invention, by fusing real-time state with future predictions, the decision-making quality and foresight of the agent are significantly improved. Compared with a single system state vector, the enhanced state representation not only includes real-time parameters such as voltage deviation and reactive power difference, but also integrates future uncertainty probability information predicted by random forest. This enables the agent to simultaneously perceive the current state of the system and the expected evolution trend. This enhanced environmental perception capability allows the Q-learning algorithm to make more predictive decisions, adjusting the coordination strategy of SVG and synchronous condenser in advance before voltage fluctuations occur. By providing comprehensive environmental information, the enhanced state representation enables the agent to better balance the relationship between rapid response and steady-state support, ensuring voltage stability and improving the economic efficiency of system operation, ultimately achieving true multi-objective collaborative optimization. Attached Figure Description

[0051] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts:

[0052] Figure 1 This is a flowchart of Embodiment 1 of the present invention;

[0053] Figure 2 This is a flowchart of Embodiment 2 of the present invention. Detailed Implementation

[0054] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0055] Example 1: Refer to Figure 1 As shown, the present invention provides a technical solution: a hybrid multi-objective coordinated control system based on SVG and synchronous phase shifter, comprising:

[0056] The real-time status monitoring module is used to continuously collect power grid operating parameters through a multi-sensor network and construct the system state space using the power grid operating parameters based on Markov decision process theory.

[0057] The reinforcement learning decision-making module takes the system state space as input, uses the Q-learning algorithm to construct a reward and punishment model, generates a reactive power compensation strategy through dynamic exploration and utilization mechanism, and outputs coordinated control commands for SVG and synchronous condenser.

[0058] The collaborative execution control module is used to receive coordinated control commands, parse and distribute action signals to the controllers of the SVG and the synchronous condenser, so as to achieve dynamic coordination of fast response and steady-state compensation and complete multi-objective optimized control.

[0059] Specifically, a real-time state monitoring module constructs a precise system state space, providing millisecond-level data support for decision-making; a reinforcement learning decision-making module autonomously generates coordination strategies based on the Q-learning algorithm, dynamically optimizing reactive power compensation commands, completely overcoming the shortcomings of traditional PID control parameters being fixed and fuzzy control rules being limited; and a collaborative execution control module accurately parses and distributes commands for execution, ensuring the organic coordination between the SVG's fast response and the synchronous condenser's steady-state compensation. Practical applications show that this system stably controls voltage deviation within ±1%, reduces response delay by 30%, and significantly improves grid voltage stability and power quality.

[0060] Furthermore, as an optimal implementation: continuously collect power grid operating parameters through a multi-sensor network, and construct the system state space based on a Markov decision process, and perform the following operations:

[0061] By utilizing voltage transformers, current transformers, and phase measurement equipment deployed at key nodes of the power grid, the original power grid operating parameters, including voltage, current, phase, reactive power, and frequency, are acquired in real time at a millisecond-level sampling frequency. The power grid operating parameters are then filtered and outlier removed.

[0062] The preprocessed power grid operating parameters are mapped into multi-dimensional state variables including voltage deviation, reactive power difference, response delay, and load fluctuation trend to construct a system state vector.

[0063] Based on Markov decision processes, the system state vector transition process is modeled as a discrete-time decision problem. Each state contains historical operating information and current state characteristics, enabling the state space to represent the dynamic evolution of the system and providing a temporally correlated state representation for reinforcement learning decision modules.

[0064] The state representation is transmitted to the reinforcement learning decision module via a protocol conversion gateway.

[0065] Specifically, through millisecond-level high-frequency acquisition and intelligent preprocessing, a precise state representation of the system's dynamics is constructed. This state representation not only includes key real-time information such as voltage deviation and reactive power difference, but also incorporates historical operating trends through a Markov decision process. This enables the reinforcement learning decision module to acquire complete environmental information that combines instantaneous state with dynamic evolution patterns. This provides a highly timely and reliable data foundation for intelligent decision-making, fundamentally solving the response mismatch problem caused by incomplete state perception and delayed decision information in traditional control. It lays the core foundation for achieving precise coordinated control of SVG and synchronous condensers.

[0066] Furthermore, as an optimal implementation: a reward and punishment model is constructed using the Q-learning algorithm, and a reactive power compensation strategy is generated through a dynamic exploration and utilization mechanism, performing the following operations:

[0067] Based on the state representation of the input, a reward function with the core elements of minimizing voltage deviation, minimizing reactive power difference and minimizing system loss is used. The reward function constructs a multi-objective optimization evaluation system through weighted summation to quantitatively evaluate the merits of the state actions selected by the built-in intelligent agent of the reinforcement learning decision module from the predefined action space.

[0068] The agent then adopts an ε-greedy strategy to balance the exploration and utilization process. By fitting the state-action value function through the Q-table and continuously updating and optimizing it, the agent is finally guided to autonomously learn a collaborative compensation strategy that takes into account both dynamic performance and static economy, so as to achieve the organic cooperation between SVG and synchronous condenser.

[0069] Specifically, by guiding the agent to autonomously learn the optimal coordination strategy through a multi-objective reward function, the problem of static strategies being unable to be dynamically optimized in traditional control is effectively solved. The reward function simultaneously optimizes voltage stability, reactive power accuracy, and system energy consumption, driving the agent to explore the optimal coordination between the SVG and the synchronous condenser in the action space. By combining an ε-greedy strategy with Q-learning, the agent gradually forms a compensation strategy that balances fast response and steady-state support during training, ultimately achieving a control effect of voltage deviation control within ±1% and a 30% reduction in response latency, significantly improving system stability and economy.

[0070] Furthermore, as an optimal implementation: the reward function constructs a multi-objective optimization evaluation system through weighted summation, and performs the following operations:

[0071] The mathematical expression for the reward function is defined as follows: in: Indicates time step The overall reward value, For the weighting coefficients, satisfying , For actual voltage measurement, The system's rated voltage. This represents the actual reactive power of the system. For the current target reactive power demand, To compensate for the total active power loss generated by the SVG and synchronous condenser during the process;

[0072] By adjusting the weighting coefficients The numerical value, dynamically configuring the preference direction of the multi-objective optimization evaluation system;

[0073] The real-time collected power grid operating parameters are substituted into the reward function calculation formula to solve for the comprehensive reward value at the current time step. ;Will As an immediate reward input to the Q-learning algorithm, it is used to update the state-action value function, driving the agent to learn towards the optimal multi-objective policy.

[0074] Specifically, complex system optimization problems are transformed into quantifiable mathematical objectives, and the optimization direction is flexibly configured through weighting coefficients. This is achieved through dynamic adjustment. The numerical value allows for flexible balancing of weight preferences among voltage stability, reactive power accuracy, and economy, enabling the system to prioritize voltage quality while also focusing on energy conservation and consumption reduction based on actual needs. The reward function maps grid parameters to a comprehensive reward value in real time, providing clear optimization direction guidance for the Q-learning algorithm. This allows the intelligent agent to accurately evaluate the comprehensive effect of each action across multiple objectives, thereby learning the optimal strategy that quickly eliminates voltage deviations while balancing reactive power compensation accuracy and operational economy, truly achieving multi-objective collaborative optimization.

[0075] Furthermore, as an optimal implementation: the agent then employs an ε-greedy strategy to balance the exploration and exploitation processes, fitting the state-action value function through a Q-table and continuously updating and optimizing it, ultimately guiding the agent to autonomously learn a collaborative compensation strategy that balances dynamic performance and static economy, and performing the following operations:

[0076] Set an initial value for the exploration rate ε and a decay strategy. Generate a random number at each decision time. When the random number is less than the current exploration rate ε, the agent randomly selects an action from the action space to explore the unknown strategy space. Otherwise, it selects the optimal action with the highest value according to the current Q table.

[0077] After performing the selected action, observe the next state and immediate reward from the environment. Calculate new state action values ​​based on the Q-learning update rules;

[0078] The corresponding entries in the Q table are updated with new state-action values ​​to gradually optimize the estimation of state-action value. At the same time, the exploration rate ε is decayed according to a preset period to gradually reduce the probability of random exploration and increase the utilization of the learned optimal strategy.

[0079] The above process is iterated until the Q-table converges and meets the predetermined requirements, so that the agent can finally obtain a stable optimal policy and adaptively output the best coordinated control command of SVG and synchronous modulator based on the real-time state representation.

[0080] Specifically, by initializing the exploration rate ε and setting a decay strategy, the agent extensively explores the action space in the early stages of training to avoid getting trapped in local optima. Subsequently, it gradually increases the proportion of existing experience utilized to ensure stable convergence of the policy. After each action is executed, the agent updates the Q-table by learning the update rule, combining immediate rewards and the optimal value of the next state, so that the estimated value of the state and action continuously approximates the true value function. This continuous iterative optimization process enables the agent to eventually obtain a stable policy that can accurately map the system state to the optimal action, thereby achieving adaptive adjustment of the SVG's fast response and the steady-state compensation capability of the camera, forming a coordinated control capability that combines dynamic performance and static economy.

[0081] Furthermore, as an optimal implementation: after executing the selected action, observe the next state and immediate reward from the environmental feedback, calculate the new state-action value according to the Q-learning update rule, and perform the following operations:

[0082] After performing the selected action, the agent first observes the next state fed back by the environment. Compared with the immediate reward calculated by the reward function Then, based on the Q-learning update rules, actions are performed according to the current state. Q value Instant rewards and the next state The maximum expected Q value of all possible actions Through calculation formula To obtain new state action values, where The learning rate is used to control the update step size. The discount factor is calculated using time-series difference error. This is used to gradually correct the estimation bias of the state action values ​​in the Q table, thereby achieving iterative optimization of the action value function.

[0083] Specifically, by utilizing real-time rewards and state transition information from environmental feedback, mathematical formulas are used to continuously correct the agent's cognitive biases regarding the value of actions. The learning rate controls the speed of knowledge updates to avoid excessive policy oscillations; a discount factor balances short-term and long-term gains to ensure long-term optimization goals. Based on the Bellman equations, the incremental learning approach enables the agent to gradually acquire accurate assessments of action value through continuous interaction with the power grid environment, ultimately forming an optimal control strategy capable of adaptively responding to complex dynamic changes in the system.

[0084] Example 2: Refer to Figure 2 As shown, unlike Embodiment 1, the system also includes a probabilistic decision-making assistance module, which is used to collect historical and real-time system state space data, perform uncertainty prediction and probabilistic representation based on the random forest algorithm, and finally output the system uncertainty prediction results to the reinforcement learning decision module. Specifically, it performs the following operations:

[0085] Historical and real-time system status data are collected, and load change rate, new energy output fluctuation trend and external environmental characteristics are extracted from them as training and prediction inputs to construct a sample set for the random forest algorithm.

[0086] The Bootstrap sampling method is used to generate multiple sub-training sets from the sample set, and a decision tree is constructed for each sub-training set. During the decision tree generation process, some features are randomly selected for node splitting.

[0087] Using the system state vector from the previous moment as input, parallel reasoning is performed through multiple decision trees in a trained random forest model, with each tree outputting a prediction of the direction of load or renewable energy power fluctuations in the short term.

[0088] The prediction results of all decision trees are statistically analyzed, and the final fluctuation direction classification is determined by voting mechanism or the probability of continuous fluctuation is calculated by mean, generating a probabilistic representation of system uncertainty.

[0089] The probabilistic representation results are used as auxiliary state variables, and the auxiliary state variables are concatenated with the system state vector to form an enhanced state representation, which is used in the decision-making process of the Q-learning algorithm.

[0090] Specifically, the probabilistic decision support module provides the system with crucial predictive capabilities. Its unique effect lies in transforming uncertainties into quantifiable decision-making criteria through the random forest algorithm. By analyzing historical and real-time state data, it accurately predicts load changes and fluctuations in renewable energy output, greatly enhancing the system's predictability of changes in the power grid environment. The ensemble learning model constructed through Bootstrap sampling and random feature selection effectively avoids overfitting and significantly improves the accuracy and robustness of the prediction results. The final probabilistic representation is fused with the system state vector to form an enhanced state representation, enabling the reinforcement learning agent to make decisions based on both the current system state and future trends. This fundamentally solves the control lag problem caused by the lack of predictability in traditional methods, making the coordination between SVG and synchronous condensers more precise and efficient.

[0091] Furthermore, as an optimal implementation: the auxiliary state variables are concatenated with the system state vector to form an enhanced state representation, and the following operations are performed:

[0092] It receives the system uncertainty prediction results from the probabilistic decision support module as auxiliary state variables, and at the same time receives the original system state vector from the real-time state monitoring module;

[0093] The auxiliary state variables and the system state vector are preprocessed to align their dimensions, ensuring that the numerical range of the two vectors matches their dimensions.

[0094] An auxiliary state variable is appended to the end of the system state vector using a vector concatenation algorithm to generate an enhanced state representation, the mathematical expression of which is:

[0095] in, This represents an enhanced state representation. Represents the original system state vector. Represents auxiliary state variables, This represents a vector concatenation operation;

[0096] The spliced ​​enhanced state representation is standardized to eliminate numerical differences caused by different physical dimensions, ensuring the convergence stability of the subsequent Q-learning algorithm.

[0097] The processed enhanced state representation is fed into the state buffer of the reinforcement learning decision module in real time, providing the Q-learning algorithm with complete environmental information including the current system state space and future uncertainty predictions.

[0098] Specifically, the challenge of utilizing multi-source information is addressed through deep data integration. By standardizing and concatenating predictive auxiliary variables with real-time state vectors, an enhanced state representation is generated that combines accuracy of the current state with predictability of future trends. This approach not only unifies data formats across different physical dimensions but also expands the agent's perception capabilities from an information perspective, enabling the Q-learning algorithm to make decisions based simultaneously on the precise current system state and probabilistic future evolution trends. This forward-looking environmental modeling method significantly improves the agent's decision-making accuracy and adaptability in uncertain power grid environments, providing a reliable information foundation for achieving multi-objective optimization control.

[0099] Furthermore, as an optimal implementation: the processed enhanced state representation is fed into the state buffer of the reinforcement learning decision module in real time, providing the Q-learning algorithm with complete environmental information including the current system state and future uncertainty predictions, and performing the following operations:

[0100] Establish a state buffer data interface protocol, define the data format and transmission frequency for enhanced state representation, and ensure compatibility with reinforcement learning decision modules;

[0101] The enhanced state representation is written into a circular queue of the state buffer through memory mapping, and the producer-consumer model is used to realize real-time data updating and reading, avoiding data access conflicts.

[0102] Add timestamps and sequence numbers to each augmented state representation in the state buffer to ensure that the Q-learning algorithm processes state data in the correct time sequence;

[0103] Based on the state verification mechanism, the integrity of the enhanced state representation stored in the buffer is verified, and abnormal or timed-out data is discarded to ensure the reliability of the environmental information input to the Q-learning algorithm.

[0104] The enhanced state representation processed in the buffer is provided to the Q-learning algorithm on demand through the API interface, enabling it to simultaneously obtain the preprocessed current system state space and the future uncertainty probability information predicted by the random forest.

[0105] Specifically, a buffer system addresses the data synchronization and timeliness issues in reinforcement learning applications. By defining a standard data interface and a circular queue structure, conflict-free reading and writing and real-time updating capabilities for high-frequency augmented state representations are ensured. A timestamp and sequence number mechanism guarantees the strict temporal order of state data, preventing decision-making errors caused by out-of-order data. Combined with a data verification mechanism, abnormal and timed-out data are effectively filtered, providing highly reliable environmental information input for the Q-learning algorithm. This enables the agent to synchronously acquire accurate current system states and verified future predictions, providing a solid data foundation for multi-objective collaborative control decisions.

[0106] The reinforcement learning decision-making module takes enhanced state representation as input, uses the Q-learning algorithm to construct a reward-penalty model, generates reactive power compensation strategies through dynamic exploration and utilization mechanisms, and outputs coordinated control commands for the SVG and synchronous condenser. Specifically, it performs the following operations:

[0107] Based on the enhanced state representation of the input, a multi-objective optimization evaluation system is constructed by weighted summation using a reward function with the core elements of minimizing voltage deviation, minimizing reactive power difference, and minimizing system loss.

[0108] The agent employs an ε-greedy strategy to balance the exploration and exploitation process, and continuously updates and optimizes the state-action value function by fitting a Q-table.

[0109] Substitute the real-time collected power grid operating parameters into the reward function calculation formula to solve for the comprehensive reward value at the current time step;

[0110] The comprehensive reward value is used as the immediate reward input of the Q-learning algorithm to update the state-action value function, driving the agent to learn towards the multi-objective optimal policy direction;

[0111] Ultimately, the intelligent agent is guided to autonomously learn a collaborative compensation strategy that balances dynamic performance and static economy, thereby achieving organic cooperation between SVG and synchronous modulator.

[0112] Specifically, by fusing real-time state with future predictions, the decision-making quality and foresight of the agent are significantly improved. Compared with the traditional single system state vector, the enhanced state representation not only includes real-time parameters such as voltage deviation and reactive power difference, but also integrates future uncertainty probability information predicted by random forests, enabling the agent to simultaneously perceive the current state of the system and the expected evolution trend. This enhanced environmental awareness allows the Q-learning algorithm to make more predictive decisions, adjusting the coordination strategy of SVG and synchronous condenser in advance before voltage fluctuations occur. By providing comprehensive environmental information, the enhanced state representation enables the agent to better balance the relationship between rapid response and steady-state support, ensuring voltage stability and improving the economics of system operation, ultimately achieving true multi-objective collaborative optimization.

[0113] The reward function constructs a multi-objective optimization evaluation system through weighted summation and performs the following operations:

[0114] The mathematical expression for the reward function is defined as follows: in: Indicates time step The overall reward value, For the weighting coefficients, satisfying , For actual voltage measurement, The system's rated voltage. This represents the actual reactive power of the system. For the current target reactive power demand, To compensate for the total active power loss caused by the SVG and synchronous condenser during the process, the weighting coefficients are adjusted. The numerical value is used to dynamically configure the preference direction of the multi-objective optimization evaluation system; the real-time collected power grid operation parameters are substituted into the reward function calculation formula to solve for the comprehensive reward value at the current time step. ;Will As an immediate reward input to the Q-learning algorithm, it is used to update the state-action value function, driving the agent to learn towards the optimal multi-objective policy.

[0115] The agent then employs an ε-greedy strategy to balance the exploration and utilization process. It fits the state-action value function using a Q-table and continuously updates and optimizes it. Ultimately, the agent is guided to autonomously learn a collaborative compensation strategy that balances dynamic performance and static economy, and performs the following operations:

[0116] Set an initial value for the exploration rate ε and a decay strategy. Generate a random number at each decision time. When the random number is less than the current exploration rate ε, the agent randomly selects an action from the action space to explore the unknown strategy space. Otherwise, it selects the optimal action with the highest value according to the current Q table.

[0117] After performing the selected action, observe the next state and immediate reward from the environment. Calculate new state action values ​​based on the Q-learning update rules;

[0118] The corresponding entries in the Q table are updated with new state-action values ​​to gradually optimize the estimation of state-action value. At the same time, the exploration rate ε is decayed according to a preset period to gradually reduce the probability of random exploration and increase the utilization of the learned optimal strategy.

[0119] The above process is iterated until the Q-table converges and meets the predetermined requirements, so that the agent can finally obtain a stable optimal policy and adaptively output the best coordinated control command of SVG and synchronous modulator based on the real-time state representation.

[0120] Furthermore, as an optimal implementation: after executing the selected action, observe the next state and immediate reward from the environmental feedback, calculate the new state-action value according to the Q-learning update rule, and perform the following operations:

[0121] After performing the selected action, the agent first observes the next state fed back by the environment. Compared with the immediate reward calculated by the reward function Then, based on the Q-learning update rules, actions are performed according to the current state. Q value Instant rewards and the next state The maximum expected Q value of all possible actions Through calculation formula To obtain new state action values, where The learning rate is used to control the update step size. The discount factor is calculated using time-series difference error. This is used to gradually correct the estimation bias of the state action values ​​in the Q table, thereby achieving iterative optimization of the action value function.

[0122] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A hybrid multi-objective coordinated control system based on SVG and synchronous condenser, characterized in that, include: The real-time status monitoring module is used to continuously collect power grid operating parameters through a multi-sensor network and construct the system state space using the power grid operating parameters based on Markov decision process theory. By continuously collecting power grid operating parameters through a multi-sensor network and constructing the system state space based on a Markov decision process, the following operations are performed: By utilizing voltage transformers, current transformers, and phase measurement equipment deployed at key nodes of the power grid, the original power grid operating parameters, including voltage, current, phase, reactive power, and frequency, are acquired in real time at a millisecond-level sampling frequency. The power grid operating parameters are then filtered and outlier removed. The preprocessed power grid operating parameters are mapped into multi-dimensional state variables including voltage deviation, reactive power difference, response delay, and load fluctuation trend to construct a system state vector. Based on Markov decision processes, the system state vector transition process is modeled as a discrete-time decision problem. Each state contains historical operating information and current state characteristics, enabling the state space to represent the dynamic evolution of the system and providing a temporally correlated state representation for reinforcement learning decision modules. The state representation is transmitted to the reinforcement learning decision module via a protocol conversion gateway; The probabilistic decision support module collects historical and real-time system state space data, performs uncertainty prediction and probabilistic representation based on the random forest algorithm, and finally outputs the system uncertainty prediction results to the reinforcement learning decision module. Collect historical and real-time system state space data, perform uncertainty prediction and probabilistic representation based on the random forest algorithm, and finally output the system uncertainty prediction results to the reinforcement learning decision module to perform the following operations: Historical and real-time system status data are collected, and load change rate, new energy output fluctuation trend and external environmental characteristics are extracted from them as training and prediction inputs to construct a sample set for the random forest algorithm. The Bootstrap sampling method is used to generate multiple sub-training sets from the sample set, and a decision tree is constructed for each sub-training set. During the decision tree generation process, some features are randomly selected for node splitting. Using the system state vector from the previous moment as input, parallel reasoning is performed through multiple decision trees in a trained random forest model, with each tree outputting a prediction of the direction of load or renewable energy power fluctuations in the short term. The prediction results of all decision trees are statistically analyzed, and the final fluctuation direction classification is determined by voting mechanism or the probability of continuous fluctuation is calculated by mean, generating a probabilistic representation of system uncertainty. The probabilistic representation results are used as auxiliary state variables, and the auxiliary state variables are concatenated with the system state vector to form an enhanced state representation, which is used in the decision-making process of the Q-learning algorithm. The reinforcement learning decision-making module uses the Q-learning algorithm to build a reward and punishment model, generates reactive power compensation strategies through dynamic exploration and utilization mechanisms, and outputs coordinated control commands for SVG and synchronous condenser. A reward and punishment model is constructed using the Q-learning algorithm. A reactive power compensation strategy is generated through a dynamic exploration and utilization mechanism, and the following operations are performed: Based on the state representation of the input, a reward function with the core elements of minimizing voltage deviation, minimizing reactive power difference and minimizing system loss is used. The reward function constructs a multi-objective optimization evaluation system through weighted summation to quantitatively evaluate the merits of the state actions selected by the built-in intelligent agent of the reinforcement learning decision module from the predefined action space. The agent then adopts an ε-greedy strategy to balance the exploration and utilization process. By fitting the state-action value function through the Q-table and continuously updating and optimizing it, the agent is finally guided to learn a collaborative compensation strategy that takes into account both dynamic performance and static economy, so as to achieve the organic cooperation between SVG and synchronous conversion. The collaborative execution control module is used to receive coordinated control commands, parse and distribute action signals to the controllers of the SVG and the synchronous condenser, so as to achieve dynamic coordination of fast response and steady-state compensation and complete multi-objective optimized control.

2. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 1, characterized in that: The reward function constructs a multi-objective optimization evaluation system through weighted summation and performs the following operations: The mathematical expression for the reward function is defined as follows: in: Indicates time step The overall reward value, For the weighting coefficients, satisfying , For actual voltage measurement, The system's rated voltage. This represents the actual reactive power of the system. For the current target reactive power demand, To compensate for the total active power loss generated by the SVG and synchronous condenser during the process; By adjusting the weighting coefficients The numerical value, dynamically configuring the preference direction of the multi-objective optimization evaluation system; The real-time collected power grid operating parameters are substituted into the reward function calculation formula to solve for the comprehensive reward value at the current time step. ;Will As an immediate reward input to the Q-learning algorithm, it is used to update the state-action value function, driving the agent to learn towards the optimal multi-objective policy.

3. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 1, characterized in that: The agent then employs an ε-greedy strategy to balance the exploration and utilization process, fitting the state-action value function using a Q-table and continuously updating and optimizing it. Ultimately, the agent is guided to autonomously learn a collaborative compensation strategy that balances dynamic performance and static economy, and performs the following operations: Set an initial value for the exploration rate ε and a decay strategy. Generate a random number at each decision time. When the random number is less than the current exploration rate ε, the agent randomly selects an action from the action space to explore the unknown strategy space. Otherwise, it selects the optimal action with the highest value according to the current Q table. After performing the selected action, observe the next state and immediate reward from the environment. Calculate new state action values ​​based on the Q-learning update rules; The corresponding entries in the Q table are updated with new state-action values ​​to gradually optimize the estimation of state-action value. At the same time, the exploration rate ε is decayed according to a preset period to gradually reduce the probability of random exploration and increase the utilization of the learned optimal strategy. The above process is iterated until the Q-table converges and meets the predetermined requirements, so that the agent can finally obtain a stable optimal policy and adaptively output the best coordinated control command of SVG and synchronous modulator based on the real-time state representation.

4. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 3, characterized in that: After executing the selected action, observe the next state and immediate reward from the environmental feedback, calculate the new state-action value according to the Q-learning update rule, and perform the following operations: After performing the selected action, the agent first observes the next state fed back by the environment. Compared with the immediate reward calculated by the reward function Then, based on the Q-learning update rules, actions are performed according to the current state. Q value Instant rewards and the next state The maximum expected Q value of all possible actions Through calculation formula To obtain new state action values, where The learning rate is used to control the update step size. The discount factor is calculated using time-series difference error. This is used to gradually correct the estimation bias of the state action values ​​in the Q table, thereby achieving iterative optimization of the action value function.

5. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 1, characterized in that: The step of concatenating the auxiliary state variables with the system state vector to form an enhanced state representation involves the following operations: It receives the system uncertainty prediction results from the probabilistic decision support module as auxiliary state variables, and at the same time receives the original system state vector from the real-time state monitoring module; The auxiliary state variables and the system state vector are preprocessed to align their dimensions, ensuring that the numerical range of the two vectors matches their dimensions. An auxiliary state variable is appended to the end of the system state vector using a vector concatenation algorithm to generate an enhanced state representation; The spliced ​​enhanced state representation is standardized to eliminate numerical differences caused by different physical dimensions, ensuring the convergence stability of the subsequent Q-learning algorithm. The processed enhanced state representation is fed into the state buffer of the reinforcement learning decision module in real time, providing the Q-learning algorithm with complete environmental information including the current system state space and future uncertainty predictions.

6. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 5, characterized in that: The mathematical expression for the vector concatenation algorithm is: in, This represents an enhanced state representation. Represents the original system state vector. Represents auxiliary state variables, This indicates a vector concatenation operation.

7. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 5, characterized in that: The processed enhanced state representation is then fed into the state buffer of the reinforcement learning decision module in real time, providing the Q-learning algorithm with complete environmental information including the current system state and future uncertainty predictions, and performing the following operations: Establish a state buffer data interface protocol and define the data format and transmission frequency for enhanced state representation; The enhanced state representation is written into a circular queue of the state buffer through memory mapping, and the producer-consumer model is used to realize real-time data updating and reading. Add timestamps and sequence numbers to each augmented state representation in the state buffer to ensure that the Q-learning algorithm processes state data in the correct time sequence; Based on the state verification mechanism, the integrity of the enhanced state representation stored in the buffer is verified, and abnormal or timed-out data is discarded. The enhanced state representation processed in the buffer is provided to the Q-learning algorithm on demand through the API interface, enabling it to simultaneously obtain the preprocessed current system state space and the future uncertainty probability information predicted by the random forest.

Images