Intelligent optimization and dispatching system for regional power grid operation mode based on deep reinforcement learning

Abstract

The present application relates to the technical field of power grid intelligent scheduling, in particular to a regional power grid operation mode intelligent optimization scheduling system based on deep reinforcement learning, comprising: dividing the benchmark and transition working condition interval from the historical operation data, and screening the target operation mode. The coupling relationship of active and reactive power of electrical elements in time sequence is analyzed by using deep reinforcement learning, and the potential contribution degree parameters of each element to the evolution of power grid state are quantitatively generated. Based on the parameters and the transition period power change gradient, the adaptability score is calculated and the scheduling seed node is selected, the multi-layer scheduling correlation graph is dynamically constructed according to the similarity of the transition period power trajectory, and the differentiated operation control instructions are dispatched according to the level and connection weight of the nodes in the graph. The present application realizes the deep mining of the potential effect of the elements and the adaptive generation of the scheduling strategy, and improves the accuracy of the scheduling decision and the collaborative efficiency of the multiple elements.

Description

Technical Field

[0001] This invention relates to the field of intelligent power grid dispatching technology, and in particular to an intelligent optimization dispatching system for regional power grid operation modes based on deep reinforcement learning. Background Technology

[0002] Optimal dispatching of regional power grids relies on in-depth analysis of historical and real-time operational data. Existing technologies typically employ methods based on fixed rules or classical optimization models to evaluate power grid operating modes and generate dispatch instructions. These methods often perform overall or simple threshold-based statistical analysis of power grid operating records to identify unfavorable operating conditions and allocate control tasks according to physical topology or empirical rules. However, traditional methods struggle to dynamically distinguish the inherent differences between baseline operation and transitional states when dealing with power grid operating conditions exhibiting complex temporal fluctuations.

[0003] Existing technical solutions have shortcomings. When assessing the impact of electrical components on the power grid state, conventional methods often consider active power, reactive power, and voltage indicators in isolation or statically, lacking quantitative analysis of the dynamic interaction and coupling relationship between active and reactive power on a continuous time scale. This leads to inaccurate assessment of the potential role of components and makes it impossible to predict their true contribution to the evolution of the system state. When forming scheduling strategies, existing technologies often rely on pre-set control unit groups or master-slave relationships, failing to start from the data itself and dynamically construct a coordinated control network based on the similarity of the actual behavior of components during critical transitions. This results in insufficient adaptability and refinement of scheduling commands, making it difficult to achieve efficient coordination of multiple components under complex operating conditions. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing an intelligent optimization scheduling system for regional power grid operation based on deep reinforcement learning.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: an intelligent optimization and scheduling system for regional power grid operation based on deep reinforcement learning, comprising: The file extraction module extracts time-series files about various operating modes from the regional power grid operation database. The time-series files record the active power trajectory, reactive power trajectory, and voltage level record of each electrical component within a continuous time window. The interval segmentation module identifies the fluctuation pattern of the active power trajectory for each operating mode and divides the complete time series into a baseline operating condition interval and a transition operating condition interval based on the identification results. The mode determination module counts the frequency of voltage levels of each electrical component exceeding a preset safety threshold within the baseline operating condition range, and, in conjunction with the start and end times of the transition operating condition range, determines the set of target operating modes to be analyzed from multiple operating modes. The contribution analysis module uses a deep reinforcement learning agent to analyze the temporal coupling relationship between the active power trajectory and reactive power trajectory of each electrical component in the target operating mode set within the baseline operating condition range, and generates the potential contribution parameters of each electrical component to the evolution of the power grid operating state. The adaptability scoring module integrates the power change gradient data of each electrical component in the transition operating condition range with the potential contribution parameter to calculate the adaptability score of each electrical component to become the regional dispatch center. The graph scheduling module selects specific electrical components as scheduling seed nodes based on the adaptability score. Based on the power trajectory similarity measurement between the scheduling seed nodes and other electrical components in the transition operating condition range, it constructs a multi-level scheduling association graph. According to the connection weight distribution between nodes at different levels, it assigns different sets of operation control commands to electrical components at different positions in the graph.

[0006] Preferably, the operation of identifying fluctuation patterns in the active power trajectory for each operating mode and dividing the complete time series into a baseline operating condition interval and a transition operating condition interval based on the identification results is specifically implemented as follows: For any given operating mode, obtain the time-series power sampling values ​​covering the entire scheduling cycle. The local variance of the power sample values ​​in the time series is calculated using a sliding time window to form a variance sequence. Detect all abrupt changes in the variance sequence that exceed a preset variance threshold, and mark the earliest abrupt change as the working condition transition anchor point; All continuous sampling periods before the operating condition transition anchor point are defined as the baseline operating condition interval, and all continuous sampling periods after the operating condition transition anchor point until the next operating condition transition anchor point or the end of the sequence are defined as the transition operating condition interval.

[0007] Preferably, the step of analyzing the temporal coupling relationship between the active power trajectory and reactive power trajectory of each electrical component in the target operating mode set within the baseline operating condition range using a deep reinforcement learning agent to generate a potential contribution parameter of each electrical component to the evolution of the power grid operating state further includes: Construct a deep reinforcement learning environment and use the active power trajectory segment and reactive power trajectory segment of each electrical component in the baseline operating condition range as the environmental state representation. Define a reward function that is negatively correlated with the absolute change in power of the electrical component at the start-up time in the transition operating condition range; Train the policy network in the deep reinforcement learning agent to output tentative actions that adjust the operating parameters of the electrical components in a given environmental state; Record the immediate reward generated by the simulated environment after each exploratory action is performed, and evaluate the long-term value of the exploratory action through a value network; The long-term value generated by each electrical component under all target operating modes is time-series smoothed and aggregated, and the normalized value of the aggregated result is used as the potential contribution parameter of the electrical component.

[0008] Preferably, the integration of the power change gradient data of each electrical component within the transition operating condition range with the potential contribution parameter to calculate the adaptability score of each electrical component as a regional dispatch center is specifically as follows: Extract the rising and falling edge data of the power trajectory of each electrical component in the transition operating range, calculate the absolute value of the average rate of change of each change edge, and use the weighted sum of the absolute values ​​of the average rate of change of all change edges as the power change gradient data of the electrical component. The potential contribution parameter is multiplied by the power change gradient data, and then multiplied by a reliability correction coefficient related to the number of historical failures of the electrical component. The product operation and the corrected result are standardized so that they fall within the range of zero to one. The standardized value is the fit score.

[0009] Preferably, the criterion for selecting a specific electrical component as a scheduling seed node based on the adaptability score is as follows: Under the same target operating mode, all electrical components are sorted in descending order according to the compatibility score; the top K percent of electrical components in the sorting are selected to form a candidate seed set; Calculate the correlation coefficient of the power trajectories of any two electrical components in the candidate seed set within the transition operating condition range. If the correlation coefficient is greater than a preset strong correlation threshold, then remove the electrical components with lower adaptability scores. The electrical components that are ultimately retained in the candidate seed set are determined as the scheduling seed nodes in the target operating mode.

[0010] Preferably, the method for constructing a multi-level scheduling association graph based on the power trajectory similarity measurement between the scheduling seed node and other electrical components in the transition operating condition range includes: For each scheduling seed node, calculate its dynamic time-warped distance from the power trajectory of all other electrical components in the power grid within the same transitional operating condition interval; The dynamic time warp distance is converted inversely and normalized to obtain the trajectory following degree of each other electrical component relative to the scheduling seed node; Multiple discrete following degree thresholds are set. Based on the relationship between the trajectory following degree and the following degree threshold, the other electrical components are divided into different levels with the scheduling seed node as the root. The electrical components with higher trajectory following degree belong to a level closer to the root node. For electrical components belonging to the same level of multiple scheduling seed nodes, they are simultaneously connected to the scheduling seed nodes, ultimately forming a scheduling association graph composed of multiple tree-like subgraphs that are interconnected with each other, starting from the scheduling seed nodes.

[0011] Preferably, the method of assigning different sets of operation control commands to electrical components at different locations in the graph based on the connection weight distribution between nodes at different levels is specifically as follows: Identify the node levels where all tree subgraphs in the scheduling association graph intersect, and mark the node levels as the core collaboration layer; Above the core collaboration layer, identify the node level with the most connected edges and mark the node level as the policy hub layer. Generate a set of operation control instructions, including fast power adjustment instructions and high-precision voltage regulation instructions, for the electrical components located between the strategy hub layer and the scheduling seed node; Generate a set of operation control instructions, including periodic power setpoint instructions and voltage range protection instructions, for electrical components located between the core collaboration layer and the strategy hub layer; Generate a set of operation control commands, including basic power maintenance commands and over-limit protection commands, for electrical components located below the core collaboration layer.

[0012] Preferably, before extracting time-series data on multiple operating modes from the regional power grid operation database, the method further includes a step of preprocessing the raw operating data: Outlier detection and removal are performed on the original power sampling sequence in the database, and the outliers are filled by linear interpolation of the effective sampling values ​​at the previous and next time steps; The imputed power sampling sequence is filtered by moving average to smooth out random measurement noise; The filtered active power trajectory, reactive power trajectory, and voltage level record are resampled and aligned according to a preset time granularity to form the time series archive.

[0013] Preferably, the policy network in the training deep reinforcement learning agent, Iterative updates are performed using a near-end strategy optimization algorithm; In each iteration, a batch of historical state-action transition data is sampled from the experience replay buffer; The value network is used to calculate the advantage function estimate for each state-action pair in the sampled data; Based on the estimated advantage function, the loss function of the policy network is constructed, and the parameters of the policy network are updated through gradient backpropagation. At the same time, the parameters of the value network are updated using the mean squared error loss function, enabling it to predict state values ​​more accurately.

[0014] Preferably, the operation of multiplying the potential contribution parameter with the power change gradient data and then multiplying by a reliability correction coefficient related to the historical failure count of the electrical component is specifically implemented as follows: Obtain the total number of fault events recorded for each electrical component within a preset historical statistical period; The total number of failure events is input into a preset reliability mapping function, which is configured to output a value that monotonically decreases as the total number of failure events increases. This value is the reliability correction coefficient. The calculated potential contribution parameter is multiplied by the power change gradient data to obtain the initial product result. The initial product result is multiplied by the reliability correction coefficient to obtain the corrected product result.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows: By using a deep reinforcement learning agent to analyze the dynamic temporal coupling relationship between the active and reactive power trajectories of each electrical component within a baseline operating range, potential contribution parameters are generated. These contribution parameters more realistically reflect the deep potential impact of components on system stability, enabling the system to identify components with hidden but crucial driving effects on operational status from historical data. This provides a deeper basis for scheduling decisions that goes beyond instantaneous measurements and rule-based judgments.

[0016] Based on the temporal similarity metric of power trajectories within the transitional operating condition range, a multi-level scheduling association graph is dynamically constructed using the component with the highest adaptability score as the seed node. This technology changes the traditional approach of generating scheduling structures based on fixed electrical topologies or preset control groups. Through data-driven similarity clustering, the system can automatically identify component clusters with coordinated behavior patterns during state transitions and naturally form multi-level coordination relationships based on connection strength. This enables precise and differentiated allocation of operational control commands based on the specific location and connection weight of nodes in the graph, realizing a shift from rigid group control to flexible adaptive coordination, and improving the coordination efficiency and targeting of control commands among multiple stakeholders in dynamic power grid processes. Attached Figure Description

[0017] Figure 1This is a timing diagram of the intelligent optimization and scheduling system for regional power grid operation based on deep reinforcement learning as described in this invention. Figure 2 A flowchart for fluctuation pattern recognition and interval segmentation operations; Figure 3 A flowchart generated for the potential contribution parameter; Figure 4 A heatmap of correlation coefficients of electrical components for the candidate seed set; Figure 5 This is a comparison diagram of the active power trajectories of the scheduling seed node and multi-level electrical components under the transitional operating condition. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0020] See Figure 1The regional power grid operation mode intelligent optimization and dispatching system based on deep reinforcement learning extracts time-series archives for various operation modes from the regional power grid operation database through an archive extraction module. These archives record the active and reactive power trajectories and voltage levels of each electrical component within a continuous time window. An interval segmentation module identifies fluctuation patterns in the active power trajectory of each operation mode and, based on the identification results, segments the complete time series into a baseline operating condition interval and a transitional operating condition interval. A mode determination module counts the frequency of voltage level records exceeding a preset safety threshold for each electrical component within the baseline operating condition interval and, combined with the start and end times of the transitional operating condition interval, determines the target operation mode set to be analyzed from multiple operation modes. A contribution analysis module uses a deep reinforcement learning agent to analyze the temporal coupling relationship between the active and reactive power trajectories of each electrical component in the target operation mode set within the baseline operating condition interval, generating potential contribution parameters for each electrical component to the evolution of the power grid operation state. A suitability scoring module integrates the power change gradient data and potential contribution parameters of each electrical component within the transitional operating condition interval to calculate the suitability score for each electrical component to become the regional dispatching center. The graph scheduling module selects specific electrical components as scheduling seed nodes based on the adaptability score. Based on the power trajectory similarity measurement between the scheduling seed node and other electrical components in the transition operating condition range, it constructs a multi-level scheduling association graph. According to the connection weight distribution between nodes at different levels, it assigns different sets of operation control commands to electrical components at different positions in the graph.

[0021] In one embodiment of the present invention, see [reference] Figure 2 Before the archive extraction module extracts time-series archives for various operating modes from the regional power grid operation database, it preprocesses the raw operating data. The preprocessing operation detects and removes outliers from the raw power sampling sequences in the database, and fills them in using linear interpolation of valid sample values ​​from previous and subsequent times. A moving average filter is applied to the filled power sampling sequences to smooth random measurement noise. The filtered active power trajectory, reactive power trajectory, and voltage level records are resampled and aligned according to a preset time granularity to form a time-series archive. The interval segmentation module identifies fluctuation patterns in the active power trajectory for each operating mode and, based on the identification results, segments the complete time series into a baseline operating condition interval and a transition operating condition interval. For any operating mode, it acquires the time-series power sampling values ​​covering the entire scheduling cycle, calculates the local variance of the time-series power sampling values ​​using a sliding time window, and forms a variance sequence. All abrupt changes exceeding a preset variance threshold in the variance sequence are detected, and the earliest abrupt change is marked as the operating condition transition anchor point. All continuous sampling periods before the operating condition transition anchor point are defined as the baseline operating condition interval, and all continuous sampling periods after the operating condition transition anchor point until the next operating condition transition anchor point or the end of the sequence are defined as the transition operating condition interval.

[0022] In practice, data preprocessing is a prerequisite for time-series archive extraction. Specifically, outlier detection was performed on the raw active power sampling sequence of a certain electrical component in the database over a 24-hour period. A threshold of three standard deviations plus or minus the historical mean was set, identifying and removing a significant outlier of 205.6 MW recorded at 14:30, which far exceeded the normal fluctuation range. Linear interpolation was used for imputation, calculating an imputed value of 200.0 MW using 199.8 MW recorded at 14:29 and 200.1 MW recorded at 14:31. A moving average filter with a window width of five was applied to the imputed complete sequence to smooth out random measurement noise. The filtered active power trajectory, reactive power trajectory, and voltage level records were then resampled and aligned to a one-minute time granularity, forming a time-series archive for subsequent analysis.

[0023] In some embodiments, the interval segmentation module identifies fluctuation patterns in the active power trajectory of each operating mode. For an operating mode where power fluctuations are caused by wind power integration, time-series power sampling values ​​covering the entire scheduling cycle are obtained. A sliding time window of one hour is used, with a step size of ten minutes, to calculate the local variance of the time-series power sampling values, forming a variance sequence. All abrupt changes exceeding a preset variance threshold of 15.0 in the variance sequence are detected, and the earliest abrupt change is marked as the operating condition transition anchor point. For example, in the variance sequence, the variance value corresponding to time index t=36 jumps from 8.7 to 18.2, exceeding the variance threshold, so time index t=36 is marked as the operating condition transition anchor point. All continuous sampling periods before the operating condition transition anchor point are defined as the baseline operating condition interval, and all continuous sampling periods after the operating condition transition anchor point until the next operating condition transition anchor point or the end of the sequence are defined as the transition operating condition interval.

[0024] In practice, the calculation of local variance using a sliding time window employs the following formula: in: This represents the local variance value at time index t. This represents the i-th active power sample value within the sliding window. This represents the average value of all active power samples within the sliding window. This indicates the number of sampling points contained within the sliding window. It's understandable that the variance threshold is set based on the statistical characteristics of historical operating data. It's also understandable that marking the operating condition transition anchor point signifies that the system has identified a transitional state that is significantly different from the previous stage.

[0025] In some embodiments, the fluctuation pattern identification process generates specific numerical comparisons. For example, within the baseline operating condition range, the local variance value is generally below 10.0, indicating smooth power fluctuations; while at the beginning of the transition operating condition range, the local variance value rises rapidly and remains above the variance threshold, reflecting drastic power changes. This comparison based on quantized variance provides a clear basis for distinguishing different operating condition ranges. Optionally, the preset variance threshold is not fixed and can be dynamically adjusted according to the characteristics of different operating modes. Optionally, for operating modes containing multiple significant fluctuation phases, the range segmentation module identifies multiple operating condition transition anchor points, thereby dividing the complete time series into alternating baseline operating condition ranges and transition operating condition range sequences.

[0026] In one embodiment of the present invention, see [reference] Figure 3 The contribution analysis module uses a deep reinforcement learning agent to analyze the temporal coupling relationship between the active and reactive power trajectories of each electrical component in the target operating mode set within the baseline operating condition range, generating potential contribution parameters for each electrical component to the evolution of the power grid operating state. A deep reinforcement learning environment is constructed, using the active and reactive power trajectory segments of each electrical component within the baseline operating condition range as environmental state representations. A reward function is defined, negatively correlated with the absolute change in power at the start-up time of the electrical component in the transition operating condition range. The policy network in the deep reinforcement learning agent is trained to output tentative actions that adjust the operating parameters of electrical components under a given environmental state. The policy network in the deep reinforcement learning agent is trained iteratively using a proximal policy optimization algorithm. In each iteration, a batch of historical state-action transition data is sampled from the experience replay buffer. The experience replay buffer stores a large number of tuples of historical states, actions, immediate rewards, and next state data generated by the agent's interaction with the environment. The sampled data serves as the base dataset for the current iteration's training. The dominance function estimate for each state-action pair in the sampled data is calculated using a value network. This estimate quantifies the expected additional long-term gain from taking an action in a specific state relative to the average action. The calculation of the dominance function estimate is typically based on the immediate reward sequence in the sampled data and the value network's prediction of state values, and is accomplished using a generalized dominance estimation algorithm.

[0027] Based on the calculated advantage function estimate, the loss function of the policy network is constructed. The loss function mainly consists of a policy probability ratio pruning term and a policy entropy reward term. The policy probability ratio is the ratio of the output probability of the current policy network to that of the historical policy network generating the sampled data for the same state action pair. The pruning term limits this ratio to ensure that the policy update magnitude is not too large, thus maintaining training stability. The entropy reward term encourages the policy network to maintain a certain degree of exploration, avoiding premature convergence to a local optimum. These two terms are combined with the advantage function estimate to form a complete loss function expression. The gradient of this loss function with respect to the policy network parameters is calculated using the gradient backpropagation algorithm, and the policy network parameters are updated using a gradient descent optimizer, guiding the policy network towards a direction that yields higher long-term rewards. The mean squared error loss function is used to update the parameters of the value network. The goal of the value network is to accurately predict the state value, i.e., the expected cumulative reward obtainable from a given state. The mean squared error loss function measures the difference between the value network's predicted value of states in the sampled data and the target reward value calculated from the actual reward sequence. By minimizing this mean squared error, the parameters of the value network are updated, enabling it to more accurately predict state values ​​and thus providing a more reliable estimate of the advantage function for the policy network. The parameter updates of the policy network and the value network are performed synchronously in each iteration, but based on their respective independent loss functions and gradients. After one iteration, the training process is repeated iteratively, continuously sampling new batches of data from the experience replay buffer, repeating the above steps until the policy network converges or reaches the preset number of training epochs.

[0028] In practical implementation, a deep reinforcement learning environment is constructed for a specific hydropower station power generation unit, using the contribution analysis module. The active power trajectory segment and the corresponding reactive power trajectory segment of the hydropower station power generation unit within the baseline operating condition interval, each lasting 60 minutes, are used together as the environmental state representation. The state is a vector containing 120 consecutive sampling points. The reward function is defined as follows: in: Indicates the instant reward value. This represents the absolute change in the active power of the hydropower station's generating unit at the start-up time during the simulated transition operating condition range. It is a positive scaling factor. The negative correlation between the reward function and the absolute change in power of the electrical component at the start of the transition condition means that if the trial action causes a sudden increase in power at the start of the transition, the reward value will decrease.

[0029] In some embodiments, a policy network is trained in the deep reinforcement learning agent. The policy network is a neural network containing three fully connected layers. In each iteration, 500 data points are randomly sampled from an experience replay buffer that stores 100,000 historical states, actions, rewards, and next state transition records. The advantage function estimate for each state-action pair in the sampled data is calculated using a value network. The advantage function estimate measures the advantage of a particular action relative to the average action. Based on the advantage function estimate, a loss function for the policy network is constructed, which includes a policy probability ratio clipping term and an entropy reward term. The weight parameters of the policy network are updated through gradient backpropagation. Simultaneously, the parameters of the value network are updated using a mean squared error loss function. The goal of the value network is to more accurately predict the expected long-term cumulative reward, i.e., the state value, for a given state.

[0030] In practical implementation, for a specific wind farm cluster electrical component, the deep reinforcement learning agent generates a series of tentative actions during training, such as tentatively adjusting the reactive power compensation setting of the wind farm cluster. The immediate reward generated by the simulated environment after each tentative action is recorded. For example, after an action to reduce reactive power output, the simulation system shows that the absolute change in the active power of the wind farm cluster at the start of the transitional operating condition is 15 MW, and the immediate reward calculated according to the reward function is negative. The long-term value of this tentative action is evaluated through a value network, which integrates the predicted rewards of multiple future steps and outputs a state-action value estimate. The long-term value generated by the wind farm cluster electrical component under all three target operating modes is time-series smoothed and aggregated. The aggregation operation uses a weighted average, and the normalized value of the aggregation result is used as the potential contribution parameter of the wind farm cluster electrical component; for example, the calculated parameter is 0.76.

[0031] In some embodiments, the policy network update process involves specific numerical comparisons. For example, before an update, the policy network has a probability of outputting action A for a certain state of 0.6, and the estimated advantage function value for that action is positive 0.5; after the update, the probability of the policy network outputting action A for the same state may be adjusted to 0.65. The goal of the value network update is to reduce the mean squared error between its predicted state value and the actual reward, for example, reducing the mean squared error from 0.05 to 0.03. It can be understood that the negative correlation between immediate reward and absolute power change guides the agent to learn action policies that can smooth the transition process and reduce power surges. It can be understood that temporal smoothing and aggregation of long-term value is to comprehensively analyze the performance of electrical components under different operating modes to obtain a stable contribution evaluation index. In specific implementations, the training of the deep reinforcement learning agent is an iterative process. After multiple iterations, the tentative actions generated by the policy network enable the behavior pattern of electrical components in the baseline operating range to establish a negatively correlated coupling relationship with the power change at the start of the transition operating range. This learned coupling relationship is quantified into long-term value and ultimately transformed into a potential contribution parameter of an electrical component, which identifies the strength of the component's potential impact on the evolution of the power grid's operating state.

[0032] In one embodiment of the present invention, the adaptability scoring module integrates the power change gradient data and potential contribution parameters of each electrical component within the transition operating condition range to calculate the adaptability score of each electrical component as a regional dispatching hub. The rising and falling edges of the power trajectory of each electrical component within the transition operating condition range are extracted, and the absolute value of the average rate of change for each edge is calculated. The weighted sum of the absolute values ​​of the average rates of change for all edges is used as the power change gradient data of the electrical component. The potential contribution parameters are multiplied by the power change gradient data, and then multiplied by a reliability correction coefficient related to the historical number of faults of the electrical component. The total number of fault events recorded for each electrical component within a preset historical statistical period is obtained. The total number of fault events is input into a preset reliability mapping function, which is configured to output a value that monotonically decreases as the total number of fault events increases; this value is the reliability correction coefficient. The calculated potential contribution parameters and power change gradient data are multiplied by scalar to obtain an initial product result. The initial product result is then multiplied by the reliability correction coefficient by scalar to obtain a corrected product result. The product operation and the corrected result are standardized so that they fall within the numerical range of 0 to 1. The standardized value is the fit score.

[0033] In practical implementation, for a specific electrical component of a photovoltaic power station, the rising and falling edges of the power trajectory of the component during the transition operating condition are extracted. The transition operating condition lasts for 30 minutes and includes one distinct rising edge and one falling edge. The average absolute rate of change of the rising edge is calculated to be 1.5 MW per minute, and the average absolute rate of change of the falling edge is calculated to be 1.2 MW per minute. The average absolute rates of change of the rising and falling edges are weighted and summed with a weight of 6:4, resulting in a power change gradient of 1.38 MW per minute for the photovoltaic power station electrical component. Simultaneously, the contribution analysis module generates a potential contribution parameter of 0.82 for this photovoltaic power station electrical component. A scalar multiplication operation is performed between the potential contribution parameter 0.82 and the power change gradient data 1.38, yielding an initial product result of 1.1316.

[0034] In some embodiments, the calculation of the reliability correction factor involves specific data. The total number of failure events recorded for the electrical components of the photovoltaic power station within a preset historical statistical period of one year is obtained, with a total of 3 failure events. The total number of failure events (3) is then input into a preset reliability mapping function: in: This represents the reliability correction factor. This indicates the total number of fault events. The initial attenuation coefficient is set to 0.1. The calculated reliability correction coefficient is 0.74. A scalar multiplication is performed between the initial product result 1.1316 and the reliability correction coefficient 0.74, resulting in a corrected product result of 0.837. The corrected product result is then standardized by linearly scaling the maximum and minimum values ​​of the corrected product results for all electrical components under the current operating mode, ensuring they fall within the range of 0 to 1. Assuming the maximum product result is 1.2 and the minimum product result is 0.3 under the current mode, the compatibility score of the photovoltaic power station's electrical components is calculated to be 0.44.

[0035] In some embodiments, the calculation process of power change gradient data generates specific numerical comparisons. For example, for an electrical component of a coal-fired power generator unit, whose power trajectory changes smoothly within the same transitional operating range, the calculated power change gradient data is only 0.2 MW per minute; while for an electrical component of an energy storage power station, its rapid charging and discharging results in a power change gradient data as high as 3.5 MW per minute. This comparison based on the quantified rate of change directly affects the magnitude of the subsequent initial product result. Optionally, the reliability mapping function can take the form of a piecewise linear function. Optionally, during weighted summation, the weights of the rising and falling edges can be differentiated according to their degree of influence on system stability.

[0036] In practice, standardization is a crucial step to ensure the comparability of scores. For example, assuming the corrected product of another gas turbine electrical component is 1.05, its suitability score, calculated using the same linear standardization formula, is 0.63. This illustrates that power change gradient data reflects the response speed and magnitude of the electrical component during transients. The introduction of a reliability correction coefficient is intended to incorporate the historical reliability information of the electrical component into the score. Electrical components with a higher total number of failure events have a smaller reliability correction coefficient, thus lowering the suitability score.

[0037] In practice, the integrated calculation process fuses information from multiple dimensions into a single scalar score. For a specific flexible load aggregate electrical component, its potential contribution parameter is relatively high at 0.9, its power change gradient data is moderate at 1.0 MW per minute, and its historical fault count is 0. Therefore, the reliability correction coefficient is 1, the initial product result is 0.9, and the standardized adaptability score is 0.75. Ultimately, all electrical components within the regional power grid are calculated using the same process to obtain an adaptability score within the range of 0 to 1. The score result is used for subsequent selection decisions of scheduling seed nodes.

[0038] In one embodiment of the present invention, the graph scheduling module selects specific electrical components as scheduling seed nodes based on their adaptability scores. Under the same target operating mode, all electrical components are sorted in descending order of their adaptability scores, and the top K percent of electrical components in the sorted list are selected to form a candidate seed set. The correlation coefficient of the power trajectories of any two electrical components in the candidate seed set within the transition operating condition range is calculated. If the correlation coefficient is greater than a preset strong correlation threshold, the electrical components with lower adaptability scores are removed. The electrical components that are ultimately retained in the candidate seed set are determined as the scheduling seed nodes under the target operating mode.

[0039] In practical implementation, for the analysis of a regional power grid containing ten main electrical components under a specific operating mode, all electrical components are sorted in descending order of their adaptability scores to form an ordered list. The top 100% (K) is set to 30%, therefore the top three electrical components are selected to form a candidate seed set. The candidate seed set includes electrical component A (adaptability score 0.92), electrical component B (adaptability score 0.85), and electrical component C (adaptability score 0.80). The correlation coefficient of the power trajectories of any two electrical components in the candidate seed set within the transition operating condition range is calculated, with a preset strong correlation threshold of 0.85. The correlation coefficient between the power trajectories of electrical component A and electrical component B is calculated to be 0.90, the correlation coefficient between electrical component A and electrical component C is calculated to be 0.60, and the correlation coefficient between the power trajectories of electrical component B and electrical component C is calculated to be 0.88.

[0040] In some embodiments, the power trajectory correlation coefficient is calculated using the Pearson correlation coefficient formula: in: This represents the correlation coefficient between the power trajectories of electrical component X and electrical component Y. and Let represent the power values ​​of the two electrical components at the i-th sampling time within the transition operating condition range, respectively. and These represent the average values ​​of all sampled power values ​​for the two electrical components during the transition operating condition range. This represents the total number of sampling points. A correlation coefficient greater than a preset strong correlation threshold indicates that the power change patterns of the two electrical components are highly similar.

[0041] Based on the comparison of the calculated correlation coefficients with the strong correlation threshold, the correlation coefficient between electrical component A and electrical component B (0.90) is greater than the strong correlation threshold of 0.85. Therefore, the electrical component with the lower suitability score needs to be eliminated. Electrical component A has a suitability score of 0.92, and electrical component B has a suitability score of 0.85, so electrical component B is eliminated. The correlation coefficient between electrical component B and electrical component C (0.88) is also greater than the strong correlation threshold of 0.85, so the electrical component with the lower suitability score needs to be eliminated. Electrical component B has a suitability score of 0.85, and electrical component C has a suitability score of 0.80, but electrical component B was already eliminated in the previous round of comparison and will not be compared again in this round. The correlation coefficient between electrical component A and electrical component C (0.60) is not greater than the strong correlation threshold, so both can be retained. The electrical components ultimately retained in the candidate seed set are electrical component A and electrical component C, which are then determined as the scheduling seed nodes for the target operating mode. Referring to Table 1, in order to clearly illustrate the selection process, Table 1 presents the key data and comparison results of electrical components in the candidate seed set.

[0042] Table 1: Correlation coefficients and screening results of electrical components in the candidate seed set

[0043] In some embodiments, the composition and selection of the candidate seed set involve specific comparisons. For example, in another analysis, if the top 30% of candidate electrical components include an electrical component D with a suitability score of 0.78 and an electrical component E with a suitability score of 0.75, and their correlation coefficient is as high as 0.91, exceeding the strong correlation threshold of 0.85, then the electrical component E with the lower suitability score is removed from the candidate seed set. It can be understood that sorting by suitability score in descending order ensures that the initial candidate set contains electrical components with the greatest potential for becoming a scheduling hub. It can also be understood that calculating the correlation coefficient and removing electrical components with lower scores among those with strong correlation is to avoid selecting redundant nodes with highly similar power behaviors as scheduling seeds, thereby enhancing the representativeness and diversity of the scheduling seed node set.

[0044] See Figure 4 In the process of strong correlation analysis and elimination of candidate seed sets, the heatmap visually presents the correlation coefficient distribution of the power trajectories of electrical components A, B, and C within the transition operating condition range (the strong correlation threshold is set to 0.85). The heatmap uses color gradients to map the magnitude of the correlation coefficients: dark red corresponds to a correlation coefficient of 1.0 (perfect correlation within the component itself), red areas correspond to high correlation intervals, and blue areas correspond to low correlation intervals. Specifically, the correlation coefficient between electrical component A and electrical component B is 0.93, which is above the strong correlation threshold, indicating that their power change patterns are highly similar. The correlation coefficients between electrical components A and C, and between B and C, are -0.21 and -0.31, respectively, both below the strong correlation threshold, indicating a low correlation in their power behavior. Combined with the suitability scoring rules, this heatmap can directly support the decision to eliminate candidate seeds: because A and B are strongly correlated, B, with its lower score, needs to be eliminated, ultimately retaining A and C as scheduling seed nodes, thus ensuring the diversity and representativeness of the seed set.

[0045] In one embodiment of the present invention, the graph scheduling module constructs a multi-level scheduling association graph based on the power trajectory similarity measure between the scheduling seed node and other electrical components in the transition operating condition interval. For each scheduling seed node, the dynamic time warping distance between its power trajectory and that of all other electrical components in the power grid in the same transition operating condition interval is calculated. The dynamic time warping distance is then converted inversely and normalized to obtain the trajectory following degree of each other electrical component relative to the scheduling seed node. Multiple discrete following degree thresholds are set. Based on the relationship between the trajectory following degree and the following degree threshold, other electrical components are divided into different levels rooted at the scheduling seed node. The higher the trajectory following degree, the closer the level of the electrical component is to the root node. For electrical components belonging to the same level of multiple scheduling seed nodes, they are simultaneously connected to the scheduling seed nodes, ultimately forming a scheduling association graph composed of multiple tree-like subgraphs starting from the scheduling seed nodes and interconnected with each other. According to the connection weight distribution between nodes at different levels in the scheduling association graph, different sets of operation control commands are assigned to electrical components at different positions in the graph. Identify the node level where all tree subgraphs in the scheduling association graph intersect, and mark this node level as the core coordination layer. Above the core coordination layer, identify the node level with the most connecting edges, and mark this node level as the strategy hub layer. Generate a set of operation control commands, including fast power adjustment commands and high-precision voltage regulation commands, for electrical components located between the strategy hub layer and the scheduling seed node. Generate a set of operation control commands, including periodic power setpoint commands and voltage range protection commands, for electrical components located between the core coordination layer and the strategy hub layer. Generate a set of operation control commands, including basic power maintenance commands and over-limit protection commands, for electrical components located below the core coordination layer.

[0046] In practical implementation, for a specific regional power grid operation mode, the scheduling seed nodes are determined to be electrical element Alpha and electrical element Beta. For the scheduling seed node electrical element Alpha, the dynamic time warping distance between its power trajectory and that of the other five electrical elements in the power grid within the same 30-minute transition condition interval is calculated. The formula for calculating the dynamic time warping distance is: in: This represents the dynamic time-warped distance between the power trajectories of electrical component A and electrical component B. This represents the i-th point in the power trajectory sequence of electrical component A. This represents the j-th point in the power trajectory sequence of electrical component B. This indicates the alignment path between two sequences.

[0047] In the specific implementation, the node levels where all tree subgraphs in the scheduling association graph intersect are identified. Among all subgraphs, the second level is the only level that simultaneously contains nodes from different subgraphs; therefore, the second level is marked as the core collaboration layer. Above the core collaboration layer, the node level with the most connecting edges is identified. In the first level, electrical element E is only connected to electrical element Alpha, and electrical element I is only connected to electrical element Beta. The scheduling seed node itself connects to multiple lower-level nodes, but since the scheduling seed node level is the root level, it is not included in this comparison; therefore, the first level is marked as the strategy hub layer. Based on the above level marking results, a set of operation control instructions is assigned to electrical elements at different positions in the graph. An operation control instruction set is generated for electrical elements located between the strategy hub layer and the scheduling seed node. Electrical elements E and I are located in this interval, and their obtained operation control instruction sets include fast power adjustment instructions and high-precision voltage regulation instructions. A set of operation control commands is generated for electrical components located between the core coordination layer and the strategy hub layer. Electrical components F and G are located in this interval, and their set of operation control commands includes periodic power setpoint commands and voltage range protection commands. A set of operation control commands is generated for electrical components below the core coordination layer. Electrical components H and J are located in this interval, and their set of operation control commands includes basic power maintenance commands and over-limit protection commands. It can be understood that the degree of trajectory following directly determines the hierarchical position of the electrical component in the scheduling correlation graph. The identification of the core coordination layer depends on a global analysis of the graph topology.

[0048] See Figure 5The study presents the active power (unit: MW) trajectory evolution characteristics of the dispatch seed nodes (electrical components Alpha and Beta) and various electrical components in the strategy hub layer (electrical components E and I) and the core coordination layer (electrical components F and G) within the transitional operating condition range of the regional power grid (corresponding to a 30-minute sampling period). From the perspective of trajectory fluctuation characteristics, the active power of the dispatch seed nodes (electrical components Alpha and Beta) is generally at a high level (mostly concentrated in the 9-12MW range), and the fluctuation amplitude is relatively significant, which matches its role as the dispatch center requiring response to changes in operating conditions. The power trajectories of the strategy hub layer components (E and I) are strongly correlated with those of the dispatch seed nodes, and the trajectory morphology is highly synchronized, reflecting their "rapid power adjustment" control command response characteristics. The power trajectory fluctuation amplitude of the core coordination layer components (F and G) is relatively gentle, and the trajectory following degree with the seed nodes shows a layered difference, consistent with their "periodic power setpoint and voltage range control" operation control logic. The differences in power trajectories of components at different levels in the figure intuitively reflect the hierarchical structure of the scheduling association graph constructed based on dynamic time warping distance and trajectory following degree, and also verify the constraint effect of hierarchical division and control command distribution on the power response characteristics of each component.

[0049] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A regional power grid operation mode intelligent optimization scheduling system based on deep reinforcement learning, characterized in that, Includes the following operations: The file extraction module extracts time-series files about various operating modes from the regional power grid operation database. The time-series files record the active power trajectory, reactive power trajectory, and voltage level record of each electrical component within a continuous time window. The interval segmentation module identifies the fluctuation pattern of the active power trajectory for each operating mode and divides the complete time series into a baseline operating condition interval and a transition operating condition interval based on the identification results. The mode determination module counts the frequency of voltage levels of each electrical component exceeding a preset safety threshold within the baseline operating condition range, and, in conjunction with the start and end times of the transition operating condition range, determines the set of target operating modes to be analyzed from multiple operating modes. The contribution analysis module uses a deep reinforcement learning agent to analyze the temporal coupling relationship between the active power trajectory and reactive power trajectory of each electrical component in the target operating mode set within the baseline operating condition range, and generates the potential contribution parameters of each electrical component to the evolution of the power grid operating state. The adaptability scoring module integrates the power change gradient data of each electrical component in the transition operating condition range with the potential contribution parameter to calculate the adaptability score of each electrical component to become the regional dispatch center. The graph scheduling module selects electrical components as scheduling seed nodes based on the adaptability score. Based on the power trajectory similarity measurement between the scheduling seed nodes and other electrical components in the transition operating condition range, it constructs a multi-level scheduling association graph and assigns different sets of operation control instructions to electrical components at different positions in the graph according to the connection weight distribution between nodes at different levels. The criterion for selecting electrical components as scheduling seed nodes based on the adaptability score is as follows: Under the same target operating mode, all electrical components are sorted in descending order according to the compatibility score; the top K percent of electrical components in the sorting are selected to form a candidate seed set; Calculate the correlation coefficient of the power trajectories of any two electrical components in the candidate seed set within the transition operating condition range. If the correlation coefficient is greater than a preset strong correlation threshold, then remove the electrical components with lower adaptability scores. The electrical components that are ultimately retained in the candidate seed set are determined as the scheduling seed nodes in the target operating mode.

2. The intelligent optimization scheduling system for regional power grid operation based on deep reinforcement learning according to claim 1, characterized in that, The specific implementation method for identifying the fluctuation pattern of the active power trajectory for each operating mode and dividing the complete time series into a baseline operating condition interval and a transition operating condition interval based on the identification result is as follows: For any given operating mode, obtain the time-series power sampling values ​​covering the entire scheduling cycle. The local variance of the power sample values ​​in the time series is calculated using a sliding time window to form a variance sequence. Detect all abrupt changes in the variance sequence that exceed a preset variance threshold, and mark the earliest abrupt change as the working condition transition anchor point; All continuous sampling periods before the operating condition transition anchor point are defined as the baseline operating condition interval, and all continuous sampling periods after the operating condition transition anchor point until the next operating condition transition anchor point or the end of the sequence are defined as the transition operating condition interval.

3. The intelligent optimization scheduling system for regional power grid operation based on deep reinforcement learning according to claim 2, characterized in that, The step of analyzing the temporal coupling relationship between the active power trajectory and reactive power trajectory of each electrical component in the target operating mode set within the baseline operating condition range using a deep reinforcement learning agent to generate a potential contribution parameter of each electrical component to the evolution of the power grid operating state further includes: Construct a deep reinforcement learning environment and use the active power trajectory segment and reactive power trajectory segment of each electrical component in the baseline operating condition range as the environmental state representation. Define a reward function that is negatively correlated with the absolute change in power of the electrical component at the start-up time in the transition operating condition range; Train the policy network in the deep reinforcement learning agent to output tentative actions that adjust the operating parameters of the electrical components in a given environmental state; Record the immediate reward generated by the simulated environment after each exploratory action is performed, and evaluate the long-term value of the exploratory action through a value network; The long-term value generated by each electrical component under all target operating modes is time-series smoothed and aggregated, and the normalized value of the aggregated result is used as the potential contribution parameter of the electrical component.

4. The intelligent optimization scheduling system for regional power grid operation based on deep reinforcement learning according to claim 3, characterized in that, The integration of power change gradient data of each electrical component within the transition operating condition range with the potential contribution parameter yields a suitability score for each electrical component to become a regional dispatch center. The specific calculation process is as follows: Extract the rising and falling edge data of the power trajectory of each electrical component in the transition operating range, calculate the absolute value of the average rate of change of each change edge, and use the weighted sum of the absolute values ​​of the average rate of change of all change edges as the power change gradient data of the electrical component. The potential contribution parameter is multiplied by the power change gradient data, and then multiplied by a reliability correction coefficient related to the number of historical failures of the electrical component. The product operation and the corrected result are standardized so that they fall within the range of zero to one. The standardized value is the fit score.

5. The intelligent optimization scheduling system for regional power grid operation based on deep reinforcement learning according to claim 4, characterized in that, The method for constructing a multi-level scheduling correlation graph based on the power trajectory similarity measurement between the scheduling seed node and other electrical components in the transition operating condition range includes: For each scheduling seed node, calculate its dynamic time-warped distance from the power trajectory of all other electrical components in the power grid within the same transitional operating condition interval; The dynamic time warp distance is converted inversely and normalized to obtain the trajectory following degree of each other electrical component relative to the scheduling seed node; Multiple discrete following degree thresholds are set. Based on the relationship between the trajectory following degree and the following degree threshold, the other electrical components are divided into different levels with the scheduling seed node as the root. The electrical components with higher trajectory following degree belong to a level closer to the root node. For electrical components belonging to the same level of multiple scheduling seed nodes, they are simultaneously connected to the scheduling seed nodes, ultimately forming a scheduling association graph composed of multiple tree-like subgraphs that are interconnected with each other, starting from the scheduling seed nodes.

6. The intelligent optimization scheduling system for regional power grid operation based on deep reinforcement learning according to claim 5, characterized in that, The method involves assigning different sets of operation control commands to electrical components at different locations in the graph based on the connection weight distribution between nodes at different levels. The specific assignment logic is as follows: Identify the node levels where all tree subgraphs in the scheduling association graph intersect, and mark the node levels as the core collaboration layer; Above the core collaboration layer, identify the node level with the most connected edges and mark the node level as the policy hub layer. Generate a set of operation control instructions, including fast power adjustment instructions and high-precision voltage regulation instructions, for the electrical components located between the strategy hub layer and the scheduling seed node; Generate a set of operation control instructions, including periodic power setpoint instructions and voltage range protection instructions, for electrical components located between the core collaboration layer and the strategy hub layer; Generate a set of operation control commands, including basic power maintenance commands and over-limit protection commands, for electrical components located below the core collaboration layer.

7. The intelligent optimization scheduling system for regional power grid operation based on deep reinforcement learning according to claim 1, characterized in that, Before extracting time-series data on various operating modes from the regional power grid operation database, the process also includes a preprocessing step for the raw operating data: Outlier detection and removal are performed on the original power sampling sequence in the database, and the outliers are filled by linear interpolation of the effective sampling values ​​at the previous and next time steps; The imputed power sampling sequence is filtered by moving average to smooth out random measurement noise; The filtered active power trajectory, reactive power trajectory, and voltage level record are resampled and aligned according to a preset time granularity to form the time series archive.

8. The intelligent optimization scheduling system for regional power grid operation based on deep reinforcement learning according to claim 3, characterized in that, The policy network in the training deep reinforcement learning agent. Iterative updates are performed using a near-end strategy optimization algorithm; In each iteration, a batch of historical state-action transition data is sampled from the experience replay buffer; The value network is used to calculate the advantage function estimate for each state-action pair in the sampled data; Based on the estimated advantage function, the loss function of the policy network is constructed, and the parameters of the policy network are updated through gradient backpropagation. At the same time, the parameters of the value network are updated using the mean squared error loss function, enabling it to predict state values ​​more accurately.

9. The intelligent optimization scheduling system for regional power grid operation based on deep reinforcement learning according to claim 4, characterized in that, The specific implementation of multiplying the potential contribution parameter with the power change gradient data, and then multiplying by a reliability correction coefficient related to the historical failure count of the electrical component, is as follows: Obtain the total number of fault events recorded for each electrical component within a preset historical statistical period; The total number of failure events is input into a preset reliability mapping function, which is configured to output a value that monotonically decreases as the total number of failure events increases. This value is the reliability correction coefficient. The calculated potential contribution parameter is multiplied by the power change gradient data to obtain the initial product result. The initial product result is multiplied by the reliability correction coefficient to obtain the corrected product result.

Images