Hybrid multi-target coordination control system based on SVG and synchronous phase modifier

Abstract

The invention relates to the technical field of power systems, and discloses a hybrid multi-target coordination control system based on SVG and a synchronous phase modifier, which significantly improves the decision quality and foresight of an intelligent agent by fusing a real-time state and future prediction, and improves the decision efficiency compared with a single system state vector. The enhanced state characterization not only comprises real-time parameters such as voltage deviation and reactive power difference, but also integrates future uncertainty probability information predicted by the random forest, so that the intelligent agent can simultaneously sense the current state and the expected evolution trend of the system, and the enhanced environment sensing capability enables the Q learning algorithm to make a more predictable decision, so that the method is more suitable for popularization and application. According to the method, the coordination strategy of the SVG and the phase modifier is adjusted in advance before voltage fluctuation occurs, state characterization is enhanced, comprehensive environment information is provided, an intelligent agent can better balance the relation between quick response and steady state support, the voltage stability is guaranteed, the economical efficiency of system operation is improved, and finally multi-target collaborative optimization in the true sense is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power systems, and particularly relates to a hybrid multi-target coordinated control system based on SVG and synchronous compensators. BACKGROUND

[0002] The prior art usually adopts a traditional control method to realize the cooperation of SVG and synchronous compensators. Specifically, the cooperation is realized mainly through PID control of preset parameters or fuzzy control based on a rule base. The PID controller generates a control signal through proportional, integral and differential operations according to the deviation of the controlled quantities such as voltage and reactive power from the target value, and sends the control signal to the SVG and the compensator respectively. The fuzzy control converts the input quantities such as the deviation and the rate of change of the deviation of the system into fuzzy quantities according to the fuzzy rules set by experts, and outputs relatively fixed coordinated control instructions after rule reasoning and defuzzification. The entire control process relies on the pre-adjusted parameters or rules, and the system responds to the changes in the power grid according to the fixed logic.

[0003] In view of the above and the existing related technologies, the following defects often exist: the control parameters or rule base of the traditional method are fixed and cannot adapt to the high uncertainty caused by sudden changes in load and fluctuations in new energy power in the power grid, resulting in a control blind area between the fast response of the SVG and the slow stability of the compensator, dynamic mismatch between the two, and thus problems such as voltage flicker exceeding the standard and low coordination efficiency, and the optimal collaboration cannot be achieved. SUMMARY

[0004] The technical problem to be solved by the present application is that the prior art has the disadvantage that a control blind area is generated between the fast response of the SVG and the slow stability of the compensator, and the two are dynamically mismatched. To this end, the present application provides a hybrid multi-target coordinated control system based on SVG and synchronous compensators.

[0005] To achieve the above purpose, the present application adopts the following technical scheme: a hybrid multi-target coordinated control system based on SVG and synchronous compensators, comprising: A real-time state monitoring module is used to continuously collect power grid operation parameters through a multi-sensor network, and to construct a system state space using the power grid operation parameters based on the Markov decision process theory; A probability decision assistance module is used to collect historical and real-time system state space data, to perform uncertainty prediction and probabilistic characterization based on a random forest algorithm, and to finally output the system uncertainty prediction result to a reinforcement learning decision module; A reinforcement learning decision module is used to construct a reward and punishment model using a Q-learning algorithm, to generate a reactive power compensation strategy through a dynamic exploration and utilization mechanism, and to output coordinated control instructions of the SVG and the synchronous compensator; The cooperative execution control module is configured to receive a coordination control instruction, analyze and distribute an action signal to a controller of the SVG and the synchronous compensator, realize dynamic cooperation of fast response and steady-state compensation, and complete multi-target optimization control.

[0006] Further, the power grid operation parameters are continuously collected through the multi-sensor network, and a system state space is constructed based on a Markov decision process, and the following operations are performed: The voltage transformer, the current transformer and the phase measurement device deployed at the key nodes of the power grid are used to obtain the raw power grid operation parameters of the power grid voltage, the power grid current, the power grid phase, the power grid reactive power and the power grid frequency in real time at a millisecond-level sampling frequency, and the power grid operation parameters are filtered and abnormal values are removed; The preprocessed power grid operation parameters are mapped into multi-dimensional state variables including voltage deviation, reactive power difference, response delay and load fluctuation trend, and a system state vector is constructed; Based on the Markov decision process, the system state vector transition process is modeled as a discrete time decision problem, each state includes historical operation information and current state characteristics, so that the state space has the ability to represent the dynamic evolution of the system, and provides a state representation with time correlation for the reinforcement learning decision module; The state representation is transmitted to the reinforcement learning decision module through the protocol conversion gateway.

[0007] Further, the historical and real-time system state space data are collected, the uncertainty prediction and probabilistic representation based on the random forest algorithm are performed, and the system uncertainty prediction result is finally output to the reinforcement learning decision module, and the following operations are performed: The historical and real-time system state data are collected, and the load change rate, the new energy output fluctuation trend and the external environment characteristics are extracted as training and prediction inputs to construct a sample set of the random forest algorithm; A plurality of sub-training sets are generated from the sample set by using the Bootstrap sampling method, and a decision tree is constructed for each sub-training set, and part of the features are randomly selected for node splitting during the decision tree generation process; The system state vector at the previous moment is input, and parallel reasoning is performed through the plurality of decision trees in the trained random forest model, and each tree outputs a prediction result of the fluctuation direction of the load or the new energy power in a short-term future; The prediction results of all decision trees are counted, and the final fluctuation direction classification is determined by a voting mechanism or the continuous fluctuation probability is calculated by an average value to generate a probabilistic representation of system uncertainty; The probabilistic representation result is used as an auxiliary state variable, and the auxiliary state variable and the system state vector are spliced to form an enhanced state representation, which is used in the decision process of the Q learning algorithm.

[0008] Further, the Q-learning algorithm is used to construct a reward and punishment model, and a reactive power compensation strategy is generated through a dynamic exploration and utilization mechanism, and the following operations are performed: Based on the input state representation, a reward function is used, which takes voltage deviation minimization, reactive power difference minimization, and system loss minimization as core elements. The reward function constructs a multi-objective optimization evaluation system through weighted summation, to quantitatively evaluate the pros and cons of the state action selected by the agent in the reinforcement learning decision module from the predefined action space. The agent then uses an epsilon-greedy strategy to balance the exploration and utilization process, fits the state action value function through the Q table, and continuously updates and optimizes it, ultimately guiding the agent to autonomously learn a collaborative compensation strategy that takes into account both dynamic performance and static economy, and realizes the organic cooperation of SVG and synchronous compensators.

[0009] Further, the reward function constructs a multi-objective optimization evaluation system through weighted summation, and the following operations are performed: The mathematical expression of the reward function is defined as: Wherein: represents the comprehensive reward value at time step , is a weight coefficient, satisfying , is the actual measured voltage, is the system rated voltage, is the actual reactive power of the system, is the current target reactive power demand, is the total active loss generated by SVG and synchronous compensators during the compensation process; By adjusting the value of the weight coefficient , the preference direction of the multi-objective optimization evaluation system is dynamically configured; Substitute the real-time collected grid operation parameters into the reward function calculation formula to solve the comprehensive reward value at the current time step; take as the immediate reward input of the Q-learning algorithm, to update the state action value function and drive the agent to learn towards the multi-objective optimal strategy.

[0010] Further, the agent then uses an epsilon-greedy strategy to balance the exploration and utilization process, fits the state action value function through the Q table, and continuously updates and optimizes it, ultimately guiding the agent to autonomously learn a collaborative compensation strategy that takes into account both dynamic performance and static economy, and the following operations are performed: Set the initial value and decay strategy of the exploration rate epsilon, generate a random number at each decision-making moment, when the random number is less than the current exploration rate epsilon, the agent randomly selects an action from the action space to explore the unknown strategy space, otherwise according to the current Q table selects the optimal action with the maximum value for utilization; After performing the selected action, observe the next state and immediate reward from the environment. Calculate new state action values ​​based on the Q-learning update rules; The corresponding entries in the Q table are updated with new state-action values ​​to gradually optimize the estimation of state-action value. At the same time, the exploration rate ε is decayed according to a preset period to gradually reduce the probability of random exploration and increase the utilization of the learned optimal strategy. The above process is iterated until the Q-table converges and meets the predetermined requirements, so that the agent can finally obtain a stable optimal policy and adaptively output the best coordinated control command of SVG and synchronous modulator based on the real-time state representation.

[0011] Furthermore, after executing the selected action, the next state and immediate reward fed back by the observed environment are used to calculate a new state-action value according to the Q-learning update rule, and the following operations are performed: After performing the selected action, the agent first observes the next state fed back by the environment. Compared with the immediate reward calculated by the reward function Then, based on the Q-learning update rules, actions are performed on the current state. Q value Instant rewards and the next state The maximum expected Q value of all possible actions Through calculation formula To obtain new state action values, where The learning rate is used to control the update step size. The discount factor is calculated using time-series difference error. This is used to gradually correct the estimation bias of the state action values ​​in the Q table, thereby achieving iterative optimization of the action value function.

[0012] Furthermore, the step of concatenating the auxiliary state variables with the system state vector to form an enhanced state representation involves the following operations: It receives the system uncertainty prediction results from the probabilistic decision support module as auxiliary state variables, and at the same time receives the original system state vector from the real-time state monitoring module; The auxiliary state variables and the system state vector are preprocessed to align their dimensions, ensuring that the numerical range of the two vectors matches their dimensions. An auxiliary state variable is appended to the end of the system state vector using a vector concatenation algorithm to generate an enhanced state representation; The spliced ​​enhanced state representation is standardized to eliminate numerical differences caused by different physical dimensions, ensuring the convergence stability of the subsequent Q-learning algorithm. The processed enhanced state representation is delivered to a state buffer of the reinforcement learning decision module in real time, so as to provide complete environment information including a current system state space and future uncertainty prediction for a Q learning algorithm.

[0013] Further, the vector splicing algorithm mathematical expression is: Wherein, Indicates an enhanced state representation, Indicates an original system state vector, Indicates an auxiliary state variable, Indicates a vector splicing operation.

[0014] Further, the processed enhanced state representation is delivered to a state buffer of the reinforcement learning decision module in real time, so as to provide complete environment information including a current system state and future uncertainty prediction for a Q learning algorithm, and the following operations are performed: A state buffer data interface protocol is established to define a data format and transmission frequency of the enhanced state representation; The enhanced state representation is written into a ring queue of the state buffer through a memory mapping mode, and a producer-consumer model is used to realize real-time updating and reading of data; A time stamp and a serial number mark are added to each enhanced state representation in the state buffer, so as to ensure that the state data is processed by the Q learning algorithm in a correct time sequence; Based on a state checking mechanism, the integrity of the enhanced state representation stored in the buffer is verified, and abnormal or overdue data is discarded; The processed enhanced state representation in the buffer is provided to the Q learning algorithm through an API interface as needed, so that the Q learning algorithm can simultaneously obtain the preprocessed current system state space and the future uncertainty probability information predicted by the random forest.

[0015] Technical effects and advantages of the present application: In the present application, a real-time state monitoring module is used to construct an accurate system state space to provide millisecond-level data support for decision-making; a reinforcement learning decision module generates a coordination strategy based on a Q learning algorithm to dynamically optimize reactive power compensation instructions, thereby completely overcoming the defects of fixed parameters in traditional PID control and limited rules in fuzzy control; a collaborative execution control module accurately analyzes and distributes the instructions for execution to ensure the organic cooperation of fast response of the SVG and steady-state compensation of the phase modifier, so that the voltage deviation is stably controlled within ±1%, the response delay is reduced by 30%, and the voltage stability and power quality of the power grid are significantly improved.

[0016] In the present application, by fusing real-time state and future prediction, the decision quality and foresight of the agent are significantly improved. Compared with a single system state vector, the enhanced state representation not only includes real-time parameters such as voltage deviation and reactive power difference, but also integrates the future uncertainty probability information predicted by the random forest, so that the agent can perceive the current state and expected evolution trend of the system at the same time. This enhanced environmental perception capability enables the Q learning algorithm to make more predictive decisions, adjusting the coordination strategy of SVG and phase modifier in advance before voltage fluctuation occurs. The enhanced state representation provides comprehensive environmental information, enabling the agent to better balance the relationship between fast response and steady-state support, ensuring voltage stability and improving the economy of system operation, and ultimately achieving true multi-objective collaborative optimization. BRIEF DESCRIPTION OF DRAWINGS

[0017] The disclosure of the present application will be described with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present application. In the drawings, the same reference numerals are used to refer to the same parts: Figure 1 Flowchart for Embodiment One of the Present Application; Figure 2 Flowchart for Embodiment Two of the Present Application. DETAILED DESCRIPTION

[0018] It is easy to understand that, according to the technical solution of the present application, those skilled in the art can propose a variety of structures and implementation methods that can be replaced with each other without changing the essential spirit of the present application. Therefore, the following detailed description and drawings are only exemplary descriptions of the technical solution of the present application, and should not be considered as the whole or as a limitation or restriction on the technical solution of the present application.

[0019] Embodiment One: Referring to Figure 1 As shown in the drawings, the present application provides a technical solution: a hybrid multi-objective coordination control system based on SVG and synchronous phase modifier, comprising: A real-time state monitoring module for continuously collecting power grid operation parameters through a multi-sensor network, and constructing a system state space using power grid operation parameters based on Markov decision process theory; A reinforcement learning decision module that takes the system state space as input, uses a Q learning algorithm to construct a reward and punishment model, generates a reactive power compensation strategy through a dynamic exploration and utilization mechanism, and outputs coordination control instructions for SVG and synchronous phase modifier; A collaborative execution control module for receiving coordination control instructions, analyzing and distributing action signals to the controllers of SVG and synchronous phase modifier, achieving dynamic cooperation of fast response and steady-state compensation, and completing multi-objective optimization control.

[0020] Specifically, the real-time state monitoring module is used to build a precise system state space to provide millisecond-level data support for decision-making; the reinforcement learning decision module autonomously generates a coordination strategy based on a Q-learning algorithm to dynamically optimize the reactive power compensation instruction, thereby completely overcoming the defects of fixed parameters in traditional PID control and limited rules in fuzzy control; and the collaborative execution control module accurately analyzes the instruction and distributes the execution to ensure the organic cooperation between the fast response of the SVG and the steady-state compensation of the phase modifier. The actual application shows that the voltage deviation is stably controlled within ±1%, the response delay is reduced by 30%, and the voltage stability and power quality of the power grid are significantly improved.

[0021] Further, as the optimal embodiment: the power grid operating parameters are continuously collected by a multi-sensor network, and a system state space is built based on a Markov decision process, and the following operations are performed: The voltage transformer, current transformer and phase measurement device deployed at the key nodes of the power grid are used to obtain the raw power grid operating parameters such as voltage, current, phase, reactive power and frequency at a millisecond-level sampling frequency, and the power grid operating parameters are filtered and abnormal values are removed; The preprocessed power grid operating parameters are mapped to multi-dimensional state variables including voltage deviation, reactive power difference, response delay and load fluctuation trend to build a system state vector; Based on the Markov decision process, the system state vector transfer process is modeled as a discrete time decision problem, each state contains historical operating information and current state characteristics, so that the state space has the ability to represent the dynamic evolution of the system, and provides the reinforcement learning decision module with state representation with time correlation; The state representation is transmitted to the reinforcement learning decision module through a protocol conversion gateway.

[0022] Specifically, by millisecond-level high-frequency acquisition and intelligent preprocessing, a state representation accurately representing the system dynamics is built. This state representation not only contains key real-time information such as voltage deviation and reactive power difference, but also incorporates historical operating trends through the Markov decision process, so that the reinforcement learning decision module can obtain complete environmental information with both instantaneous state and dynamic evolution law. This provides a high-time-efficiency and high-reliability data foundation for intelligent decision-making, fundamentally solving the response mismatch problem caused by incomplete state perception and lagging decision-making information in traditional control, and laying a core foundation for realizing precise collaborative control of SVG and phase modifier.

[0023] Further, as the optimal embodiment: a Q-learning algorithm is used to build a reward and punishment model to generate a reactive power compensation strategy through a dynamic exploration and utilization mechanism, and the following operations are performed: Based on the state representation of the input, a reward function with the core elements of minimizing voltage deviation, minimizing reactive power difference and minimizing system loss is used. The reward function constructs a multi-objective optimization evaluation system through weighted summation to quantitatively evaluate the merits of the state actions selected by the built-in intelligent agent of the reinforcement learning decision module from the predefined action space. The agent then adopts an ε-greedy strategy to balance the exploration and utilization process. By fitting the state-action value function through the Q-table and continuously updating and optimizing it, the agent is finally guided to autonomously learn a collaborative compensation strategy that takes into account both dynamic performance and static economy, so as to achieve the organic cooperation between SVG and synchronous condenser.

[0024] Specifically, by guiding the agent to autonomously learn the optimal coordination strategy through a multi-objective reward function, the problem of static strategies being unable to be dynamically optimized in traditional control is effectively solved. The reward function simultaneously optimizes voltage stability, reactive power accuracy, and system energy consumption, driving the agent to explore the optimal coordination between the SVG and the synchronous condenser in the action space. By combining an ε-greedy strategy with Q-learning, the agent gradually forms a compensation strategy that balances fast response and steady-state support during training, ultimately achieving a control effect of voltage deviation control within ±1% and a 30% reduction in response latency, significantly improving system stability and economy.

[0025] Furthermore, as an optimal implementation: the reward function constructs a multi-objective optimization evaluation system through weighted summation, and performs the following operations: The mathematical expression for the reward function is defined as follows: in: Indicates time step The overall reward value, Let be the weighting coefficient, satisfying , For actual voltage measurement, The system's rated voltage. This represents the actual reactive power of the system. For the current target reactive power demand, To compensate for the total active power loss generated by the SVG and synchronous condenser during the process; By adjusting the weighting coefficients The numerical value, dynamically configuring the preference direction of the multi-objective optimization evaluation system; The real-time collected power grid operating parameters are substituted into the reward function calculation formula to solve for the comprehensive reward value at the current time step. ;Will As an immediate reward input to the Q-learning algorithm, it is used to update the state-action value function, driving the agent to learn towards the optimal multi-objective policy.

[0026] Specifically, complex system optimization problems are transformed into quantifiable mathematical objectives, and the optimization direction is flexibly configured through weighting coefficients. This is achieved through dynamic adjustment. The numerical value of the reward function can flexibly balance the weight preferences among voltage stability, reactive power accuracy and economy, so that the system can prioritize the guarantee of voltage quality and focus on energy saving and consumption reduction according to actual demand. The reward function maps the grid parameters into a comprehensive reward value in real time, provides a clear optimization direction guide for the Q-learning algorithm, and enables the agent to accurately evaluate the comprehensive effect of each action in multiple target dimensions, so as to learn the optimal strategy that can eliminate voltage deviation quickly while considering the reactive power compensation accuracy and operation economy, and truly realize the multi-objective collaborative optimization.

[0027] Furthermore, as the optimal embodiment: the agent then adopts the ε-greedy strategy to balance the exploration and utilization process, fits the state-action value function through the Q table and continuously updates the optimization, and finally guides the agent to learn the collaborative compensation strategy considering dynamic performance and static economy, and performs the following operations: Set the initial value and decay strategy of the exploration rate ε, generate a random number at each decision-making moment, and when the random number is less than the current exploration rate ε, the agent randomly selects an action from the action space to explore the unknown strategy space, otherwise selects the optimal action with the maximum value according to the current Q table for utilization; Observe the next state and immediate reward of the environment feedback after executing the selected action According to the Q-learning update rule, calculate the new state-action value; Update the corresponding entry in the Q table with the new state-action value, gradually optimize its estimation of the state-action value, and gradually reduce the random exploration probability and increase the utilization of the learned optimal strategy according to the preset period of decay of the exploration rate ε; Continue the above process until the Q table converges to meet the predetermined requirements, so that the agent finally obtains a stable optimal strategy and can adaptively output the best coordinated control instructions of the SVG and the synchronous compensator according to the real-time state representation.

[0028] Specifically, by initializing the exploration rate ε and setting the decay strategy, the agent extensively explores the action space in the early stage of training to avoid falling into a local optimum, and then gradually increases the proportion of utilizing existing experience to ensure the stable convergence of the strategy. After each action execution, the agent updates the Q table through the Q-learning update rule combined with the immediate reward and the optimal value of the next state, so that the state-action value estimation continuously approaches the true value function. This continuous iterative optimization process enables the agent to finally obtain a stable strategy that can accurately map the system state and the optimal action, thereby realizing the adaptive deployment of the SVG fast response and the synchronous compensator steady-state compensation capability, and forming a coordinated control capability with dynamic performance and static economy.

[0029] Furthermore, as the optimal embodiment: after executing the selected action, the next state and immediate reward of the environment feedback are observed, the new state-action value is calculated according to the Q-learning update rule, and the following operations are performed: After performing the selected action, the agent first observes the next state fed back by the environment. Compared with the immediate reward calculated by the reward function Then, based on the Q-learning update rules, actions are performed on the current state. Q value Instant rewards and the next state The maximum expected Q value of all possible actions Through calculation formula To obtain new state action values, where The learning rate is used to control the update step size. The discount factor is calculated using time-series difference error. This is used to gradually correct the estimation bias of the state action values ​​in the Q table, thereby achieving iterative optimization of the action value function.

[0030] Specifically, by utilizing real-time rewards and state transition information from environmental feedback, mathematical formulas are used to continuously correct the agent's cognitive biases regarding the value of actions. The learning rate controls the speed of knowledge updates to avoid excessive policy oscillations; a discount factor balances short-term and long-term gains to ensure long-term optimization goals. Based on the Bellman equations, the incremental learning approach enables the agent to gradually acquire accurate assessments of action value through continuous interaction with the power grid environment, ultimately forming an optimal control strategy capable of adaptively responding to complex dynamic changes in the system.

[0031] Example 2: Refer to Figure 2 As shown, unlike Embodiment 1, the system also includes a probabilistic decision-making assistance module, which is used to collect historical and real-time system state space data, perform uncertainty prediction and probabilistic representation based on the random forest algorithm, and finally output the system uncertainty prediction results to the reinforcement learning decision module. Specifically, it performs the following operations: Historical and real-time system status data are collected, and load change rate, new energy output fluctuation trend and external environmental characteristics are extracted from them as training and prediction inputs to construct a sample set for the random forest algorithm. The Bootstrap sampling method is used to generate multiple sub-training sets from the sample set, and a decision tree is constructed for each sub-training set. During the decision tree generation process, some features are randomly selected for node splitting. Using the system state vector from the previous moment as input, parallel reasoning is performed through multiple decision trees in the trained random forest model, with each tree outputting a prediction of the direction of load or renewable energy power fluctuations in the short term. The prediction results of all decision trees are statistically analyzed, and the final fluctuation direction classification is determined by voting mechanism or the probability of continuous fluctuation is calculated by mean, generating a probabilistic representation of system uncertainty. The probabilistic representation result is taken as an auxiliary state variable, and the auxiliary state variable is spliced with a system state vector to form an enhanced state representation for a decision-making process of a Q learning algorithm.

[0032] Specifically, the probability decision auxiliary module provides key prediction capabilities for the system, and its unique effect is to convert uncertainty factors into quantifiable decision-making basis through a random forest algorithm. By analyzing historical and real-time state data, the load change and new energy output fluctuation trend are accurately predicted, which greatly enhances the predictability of the system to the changes in the power grid environment. Through Bootstrap sampling and random feature selection, the ensemble learning model effectively avoids overfitting problems and significantly improves the accuracy and robustness of the prediction results. The final output of the probabilistic representation is fused with the system state vector to form an enhanced state representation, so that the reinforcement learning agent can make decisions based on the current system state and future trend, fundamentally solving the control lag problem caused by the lack of predictability of traditional methods, and making the cooperation of SVG and synchronous compensators more accurate and efficient.

[0033] Further, as the optimal embodiment: the auxiliary state variable is spliced with the system state vector to form an enhanced state representation, and the following operations are performed: receiving the system uncertainty prediction result from the probability decision auxiliary module as an auxiliary state variable, and receiving the original system state vector from the real-time state monitoring module; dimensionally aligning and preprocessing the auxiliary state variable and the system state vector to ensure that the numerical ranges and dimensions of the two vectors match; appending the auxiliary state variable to the end of the system state vector using a vector splicing algorithm to generate an enhanced state representation, and the mathematical expression is:

[0034] wherein, represents the enhanced state representation, represents the original system state vector, represents the auxiliary state variable, represents a vector splicing operation; standardizing the spliced enhanced state representation to eliminate numerical differences caused by different physical dimensions, and ensuring the convergence stability of the subsequent Q learning algorithm; delivering the processed enhanced state representation to the state buffer of the reinforcement learning decision-making module in real time to provide complete environmental information containing the current system state space and future uncertainty prediction for the Q learning algorithm.

[0035] Specifically, the multi-source information utilization problem is solved by deep integration at the data level. By standardizing and splicing the predictive auxiliary variables and real-time state vectors, an enhanced state representation is generated, which has both current state accuracy and future trend predictability. This processing method not only unifies the data format of different physical dimensions, but also expands the perception ability of the agent from the information dimension, so that the Q learning algorithm can make decisions based on both the accurate current state of the system and the probabilistic future evolution trend. This forward-looking environmental modeling method significantly improves the decision accuracy and adaptability of the agent in the uncertain environment of the power grid, providing a reliable information foundation for multi-objective optimization control.

[0036] Furthermore, as the optimal embodiment: the processed enhanced state representation is delivered to the state buffer of the reinforcement learning decision module in real time, providing the Q learning algorithm with complete environmental information including the current system state and future uncertainty prediction, and performing the following operations: Establish a state buffer data interface protocol to define the data format and transmission frequency of the enhanced state representation, ensuring compatibility with the reinforcement learning decision module; Write the enhanced state representation into the ring queue of the state buffer through memory mapping, and use the producer-consumer model to realize real-time updating and reading of data, avoiding data access conflicts; Add a timestamp and a sequence number to each enhanced state representation in the state buffer to ensure that the Q learning algorithm processes the state data in the correct time sequence; Based on the state verification mechanism, the integrity of the enhanced state representation stored in the buffer is verified, and abnormal or timeout data is discarded to ensure the reliability of the environmental information input to the Q learning algorithm; Through the API interface, the processed enhanced state representation in the buffer is provided to the Q learning algorithm as needed, so that it can simultaneously obtain the preprocessed current system state space and the future uncertainty probability information predicted by the random forest.

[0037] Specifically, the data synchronization and timeliness problem of reinforcement learning in engineering application is solved by the buffer. By defining a standard data interface and a ring queue structure, the non-conflict reading and writing and real-time updating capability of high-frequency enhanced state representation are ensured. The timestamp and sequence number mechanism guarantees the strict time sequence of the state data, avoiding decision logic errors caused by out-of-order data. Combined with the data verification mechanism, abnormal and timeout data are effectively filtered, providing highly reliable environmental information input to the Q learning algorithm. The agent can simultaneously obtain accurate current system state and verified future prediction information, providing a solid data support foundation for multi-objective collaborative control decision.

[0038] The reinforcement learning decision module takes the enhanced state representation as input, uses the Q-learning algorithm to construct a reward and punishment model, generates a reactive power compensation strategy through a dynamic exploration and utilization mechanism, and outputs coordinated control instructions for SVG and synchronous compensators. Specifically, the following operations are performed: Based on the input enhanced state representation, a reward function is used, which minimizes voltage deviation, minimizes reactive power difference, and minimizes system loss as core elements, and a multi-objective optimization evaluation system is constructed by weighted summation; The agent uses an epsilon-greedy strategy to balance exploration and utilization, fits the state-action value function through a Q-table, and continuously updates and optimizes it; The real-time collected grid operating parameters are substituted into the reward function calculation formula to solve the comprehensive reward value at the current time step; The comprehensive reward value is used as the immediate reward input of the Q-learning algorithm to update the state-action value function and drive the agent to learn towards the multi-objective optimal strategy; Finally, the agent autonomously learns a collaborative compensation strategy that takes into account both dynamic performance and static economy, achieving the organic coordination of SVG and synchronous compensators.

[0039] Specifically, by fusing real-time state and future prediction, the decision-making quality and foresight of the agent are significantly improved. Compared with the traditional single system state vector, the enhanced state representation not only includes real-time parameters such as voltage deviation and reactive power difference, but also integrates the future uncertainty probability information predicted by the random forest, enabling the agent to perceive both the current state and the expected evolution trend of the system. This enhanced environmental perception capability enables the Q-learning algorithm to make more forward-looking decisions, adjusting the coordination strategy of SVG and synchronous compensators in advance before voltage fluctuations occur. The enhanced state representation provides comprehensive environmental information, enabling the agent to better balance the relationship between fast response and steady-state support, ensuring voltage stability while improving system operation economy, ultimately achieving true multi-objective collaborative optimization.

[0040] The reward function constructs a multi-objective optimization evaluation system by weighted summation, and the following operations are performed: The mathematical expression of the reward function is defined as: Wherein: represents the comprehensive reward value at time step , is the weight coefficient, satisfying , is the actual measured voltage, is the system rated voltage, is the actual reactive power of the system, is the current target reactive power demand, is the total active loss generated by SVG and synchronous compensators during compensation; by adjusting the weight coefficient the numerical value of the dynamic configuration of multi-objective optimization evaluation system preference direction; the real-time acquisition of power grid operation parameters into the reward function calculation formula, the solution of the current time step of the comprehensive reward value ; as the Q learning algorithm immediate reward input, for updating the state action value function, driving the agent to learn to the multi-objective optimal strategy direction.

[0041] The agent then uses the epsilon greedy strategy to balance exploration and utilization, fits the state action value function through the Q table and continuously updates the optimization, and finally guides the agent to learn the collaborative compensation strategy considering dynamic performance and static economy, and performs the following operations: Set the initial value of the exploration rate ε and the attenuation strategy, generate a random number at each decision-making moment, when the random number is less than the current exploration rate ε, the agent randomly selects an action from the action space to explore the unknown policy space, otherwise according to the current Q table selects the optimal action with the maximum value to utilize; Observe the next state and immediate reward feedback after executing the selected action , according to the Q learning update rule, calculate the new state action value; Update the corresponding entry in the Q table with the new state action value, gradually optimize its estimation of the state action value, and gradually reduce the random exploration probability and increase the utilization of the learned optimal strategy according to the preset period of time attenuation exploration rate ε; Continue the above process until the Q table converges to meet the predetermined requirements, so that the agent finally obtains a stable optimal strategy and can adaptively output the best coordinated control instructions of SVG and synchronous compensator according to the real-time state representation.

[0042] Further, as the optimal embodiment: after executing the selected action, observing the next state and immediate reward feedback from the environment, according to the Q learning update rule, calculating the new state action value, performing the following operations: After executing the selected action, the agent first observes the next state and the immediate reward calculated by the reward function , then according to the Q learning update rule, based on the Q value of the current state action , the maximum expected Q value of all possible actions in the next state , through the calculation formula to get the new state action value, where is the learning rate used to control the update step size, is the discount factor, the calculation process through the time difference error The state-action value estimation bias in the Q table is gradually corrected, so as to realize iterative optimization of the action value function.

[0043] The technical scope of the present application is not limited to the above description, and those skilled in the art can make various modifications and changes to the above embodiments without departing from the technical idea of the present application, and these modifications and changes shall all belong to the protection scope of the present application.

Claims

1. A hybrid multi-objective coordinated control system based on SVG and synchronous condenser, characterized in that, include: The real-time status monitoring module is used to continuously collect power grid operating parameters through a multi-sensor network and construct the system state space using the power grid operating parameters based on Markov decision process theory. The probabilistic decision support module collects historical and real-time system state space data, performs uncertainty prediction and probabilistic representation based on the random forest algorithm, and finally outputs the system uncertainty prediction results to the reinforcement learning decision module. The reinforcement learning decision-making module uses the Q-learning algorithm to build a reward and punishment model, generates reactive power compensation strategies through dynamic exploration and utilization mechanisms, and outputs coordinated control commands for SVG and synchronous condenser. The collaborative execution control module is used to receive coordinated control commands, parse and distribute action signals to the controllers of the SVG and the synchronous condenser, realize the dynamic coordination of fast response and steady-state compensation, and complete multi-objective optimized control.

2. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 1, characterized in that: The system continuously collects power grid operating parameters through a multi-sensor network and constructs the system state space based on a Markov decision process, then performs the following operations: By utilizing voltage transformers, current transformers, and phase measurement equipment deployed at key nodes of the power grid, the original power grid operating parameters, including voltage, current, phase, reactive power, and frequency, are acquired in real time at a millisecond-level sampling frequency. The power grid operating parameters are then filtered and outlier removed. The preprocessed power grid operating parameters are mapped into multi-dimensional state variables including voltage deviation, reactive power difference, response delay, and load fluctuation trend to construct a system state vector. Based on Markov decision processes, the system state vector transition process is modeled as a discrete-time decision problem. Each state contains historical operating information and current state characteristics, enabling the state space to represent the dynamic evolution of the system and providing a temporally correlated state representation for reinforcement learning decision modules. The state representation is transmitted to the reinforcement learning decision module via a protocol conversion gateway.

3. The hybrid multi-objective coordinated control system based on SVG and synchronous phase shifter according to claim 1 or 2, characterized in that: The collected historical and real-time system state space data are used to perform uncertainty prediction and probabilistic representation based on the random forest algorithm. Finally, the system uncertainty prediction results are output to the reinforcement learning decision module to perform the following operations: Historical and real-time system status data are collected, and load change rate, new energy output fluctuation trend and external environmental characteristics are extracted from them as training and prediction inputs to construct a sample set for the random forest algorithm. The Bootstrap sampling method is used to generate multiple sub-training sets from the sample set, and a decision tree is constructed for each sub-training set. During the decision tree generation process, some features are randomly selected for node splitting. Using the system state vector from the previous moment as input, parallel reasoning is performed through multiple decision trees in the trained random forest model, with each tree outputting a prediction of the direction of load or renewable energy power fluctuations in the short term. The prediction results of all decision trees are statistically analyzed, and the final fluctuation direction classification is determined by voting mechanism or the probability of continuous fluctuation is calculated by mean, generating a probabilistic representation of system uncertainty. The probabilistic representation results are used as auxiliary state variables, and the auxiliary state variables are concatenated with the system state vector to form an enhanced state representation, which is used in the decision-making process of the Q-learning algorithm.

4. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 3, characterized in that: The Q-learning algorithm is used to construct a reward and punishment model, and a reactive power compensation strategy is generated through a dynamic exploration and utilization mechanism to perform the following operations: Based on the state representation of the input, a reward function with the core elements of minimizing voltage deviation, minimizing reactive power difference and minimizing system loss is used. The reward function constructs a multi-objective optimization evaluation system through weighted summation to quantitatively evaluate the merits of the state actions selected by the built-in intelligent agent of the reinforcement learning decision module from the predefined action space. The agent then adopts an ε-greedy strategy to balance the exploration and utilization process. By fitting the state-action value function through the Q-table and continuously updating and optimizing it, the agent is finally guided to autonomously learn a collaborative compensation strategy that takes into account both dynamic performance and static economy, so as to achieve the organic cooperation between SVG and synchronous condenser.

5. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 4, characterized in that: The reward function constructs a multi-objective optimization evaluation system through weighted summation and performs the following operations: The mathematical expression for the reward function is defined as follows: in: Indicates time step The overall reward value, Let be the weighting coefficient, satisfying , For actual voltage measurement, The system's rated voltage. This represents the actual reactive power of the system. For the current target reactive power demand, To compensate for the total active power loss generated by the SVG and synchronous condenser during the process; By adjusting the weighting coefficients The numerical value, dynamically configuring the preference direction of the multi-objective optimization evaluation system; The real-time collected power grid operating parameters are substituted into the reward function calculation formula to solve for the comprehensive reward value at the current time step. ;Will As an immediate reward input to the Q-learning algorithm, it is used to update the state-action value function, driving the agent to learn towards the optimal multi-objective policy.

6. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 4, characterized in that: The agent then employs an ε-greedy strategy to balance the exploration and utilization process. It fits the state-action value function using a Q-table and continuously updates and optimizes it. Ultimately, the agent is guided to autonomously learn a collaborative compensation strategy that balances dynamic performance and static economy, and performs the following operations: Set an initial value for the exploration rate ε and a decay strategy. Generate a random number at each decision time. When the random number is less than the current exploration rate ε, the agent randomly selects an action from the action space to explore the unknown strategy space. Otherwise, it selects the optimal action with the highest value according to the current Q table. After performing the selected action, observe the next state and immediate reward from the environment. Calculate new state action values ​​based on the Q-learning update rules; The corresponding entries in the Q table are updated with new state-action values ​​to gradually optimize the estimation of state-action value. At the same time, the exploration rate ε is decayed according to a preset period to gradually reduce the probability of random exploration and increase the utilization of the learned optimal strategy. The above process is iterated until the Q-table converges and meets the predetermined requirements, so that the agent can finally obtain a stable optimal policy and adaptively output the best coordinated control command of SVG and synchronous modulator based on the real-time state representation.

7. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 6, characterized in that: After executing the selected action, observe the next state and immediate reward from the environmental feedback, calculate the new state-action value according to the Q-learning update rule, and perform the following operations: After performing the selected action, the agent first observes the next state fed back by the environment. Compared with the immediate reward calculated by the reward function Then, based on the Q-learning update rules, actions are performed on the current state. Q value Instant rewards and the next state The maximum expected Q value of all possible actions Through calculation formula To obtain new state action values, where The learning rate is used to control the update step size. The discount factor is calculated using time-series difference error. This is used to gradually correct the estimation bias of the state action values ​​in the Q table, thereby achieving iterative optimization of the action value function.

8. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 3, characterized in that: The step of concatenating the auxiliary state variables with the system state vector to form an enhanced state representation involves the following operations: It receives the system uncertainty prediction results from the probabilistic decision support module as auxiliary state variables, and at the same time receives the original system state vector from the real-time state monitoring module; The auxiliary state variables and the system state vector are preprocessed to align their dimensions, ensuring that the numerical range of the two vectors matches their dimensions. An auxiliary state variable is appended to the end of the system state vector using a vector concatenation algorithm to generate an enhanced state representation; The spliced ​​enhanced state representation is standardized to eliminate numerical differences caused by different physical dimensions, ensuring the convergence stability of the subsequent Q-learning algorithm. The processed enhanced state representation is fed into the state buffer of the reinforcement learning decision module in real time, providing the Q-learning algorithm with complete environmental information including the current system state space and future uncertainty predictions.

9. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 8, characterized in that: The mathematical expression for the vector concatenation algorithm is: in, This represents an enhanced state representation. Represents the original system state vector. Represents auxiliary state variables, This indicates a vector concatenation operation.

10. The hybrid multi-objective coordinated control system based on SVG and synchronous condenser according to claim 8, characterized in that: The processed enhanced state representation is then fed into the state buffer of the reinforcement learning decision module in real time, providing the Q-learning algorithm with complete environmental information including the current system state and future uncertainty predictions, and performing the following operations: Establish a state buffer data interface protocol and define the data format and transmission frequency for enhanced state representation; The enhanced state representation is written into a circular queue of the state buffer through memory mapping, and the producer-consumer model is used to realize real-time data updating and reading. Add timestamps and sequence numbers to each augmented state representation in the state buffer to ensure that the Q-learning algorithm processes state data in the correct time sequence; Based on the state verification mechanism, the integrity of the enhanced state representation stored in the buffer is verified, and abnormal or timed-out data is discarded. The enhanced state representation processed in the buffer is provided to the Q-learning algorithm on demand through the API interface, enabling it to simultaneously obtain the preprocessed current system state space and the future uncertainty probability information predicted by the random forest.

Images