An intelligent regulation and control system for power transformers based on deep reinforcement learning

Abstract

The application discloses a kind of based on depth reinforcement learning's electric power transformer intelligent regulation and control system, comprising: data acquisition and trajectory generation module, for generating operation trajectory;Trajectory fracture identification module, for trajectory topology fracture identification is formed trajectory fracture position set;Strategy space construction module, for constructing trajectory reconstruction strategy space, obtain candidate reconstruction trajectory set;Joint representation module, for operation trajectory and candidate reconstruction trajectory set input improved IQL, generate trajectory feature representation;Constraint propagation module, for establishing path internal self-constraint propagation relationship, obtain constraint consistent candidate reconstruction trajectory set;Strategy optimization module, for value evaluation, output optimal trajectory reconstruction strategy;Regulation and control execution module, for obtaining electric power transformer intelligent regulation and control result.The application adopts trajectory level depth reinforcement learning method, realizes transformer collaborative regulation and control, with the advantages of global consistency is strong and decision stability is high.

Description

Technical Field

[0001] This invention relates to the field of intelligent control of power transformers, and in particular to an intelligent control system for power transformers based on deep reinforcement learning. Background Technology

[0002] With the continuous improvement of the intelligence level of power systems, the monitoring and control of the operating status of power transformers gradually rely on multi-source time-series data collected by SCADA systems. Data analysis to achieve operating status assessment and control decisions has become the mainstream technical path. Existing technologies usually model the state characteristics based on a single point in time or a short time window, and combine traditional control strategies or reinforcement learning methods to adjust the operating status of transformers in order to achieve safe and stable operation.

[0003] Most existing methods ignore the overall evolutionary characteristics of the trajectory and lack the ability to identify and reconstruct abnormal trajectory fracture structures. At the same time, in the reinforcement learning modeling process, the state and action are usually treated as independent variables, without considering the correlation between different time points within the trajectory, and without introducing the difference constraints between the running trajectory and the candidate trajectory. This makes the policy optimization process susceptible to local fluctuations, making it difficult to achieve global consistency control, and thus affecting the stability and reliability of the control results. Summary of the Invention

[0004] One objective of this invention is to propose an intelligent control system for power transformers based on deep reinforcement learning. This invention employs a trajectory-level deep reinforcement learning method to achieve coordinated control of transformers, possessing the advantages of strong global consistency and high decision stability.

[0005] According to an embodiment of the present invention, a power transformer intelligent control system based on deep reinforcement learning includes:

[0006] The data acquisition and trajectory generation module is used to acquire SCADA multi-source operation data of power transformers and preprocess it to generate operation trajectories;

[0007] The trajectory breakage identification module is used to identify trajectory topological breaks using the running trajectory, extract abnormal change segments, and form a set of trajectory breakage locations.

[0008] The strategy space construction module is used to construct a trajectory reconstruction strategy space based on the set of trajectory breakage locations, and to convert each trajectory reconstruction strategy into a candidate reconstruction trajectory to obtain a set of candidate reconstruction trajectories.

[0009] The joint characterization module is used to input the running trajectory and the candidate reconstructed trajectory set into the improved IQL, perform joint characterization on the running trajectory and the candidate reconstructed trajectory, and generate trajectory feature representation;

[0010] The constraint propagation module is used to establish a path-internal self-constraint propagation relationship for the control actions in the candidate reconstruction trajectory within the improved IQL, and to perform coordinated adjustments on each control action based on the propagation relationship to obtain a set of constraint-consistent candidate reconstruction trajectories.

[0011] The strategy optimization module is used to evaluate the value of each trajectory reconstruction strategy in the improved IQL based on the constraint-consistent candidate reconstruction trajectory set, using the trajectory reconstruction strategy space as the action space, and to complete the strategy optimization and output the optimal trajectory reconstruction strategy.

[0012] The control execution module is used to deduce the corresponding control action sequence based on the optimal trajectory reconstruction strategy, generate power transformer control instructions, and obtain intelligent control results for the power transformer.

[0013] Optionally, the preprocessing specifically includes data synchronization and alignment, abnormal data removal, missing data completion, multi-source data normalization, temporal resampling, state feature construction, and continuous trajectory stitching.

[0014] Optionally, the trajectory breakage recognition module includes:

[0015] The operating trajectory is obtained, which consists of operating state vectors arranged in chronological order. Each operating state vector includes voltage, current, temperature and load parameters.

[0016] Perform differential calculation on adjacent running state vectors in the running trajectory to generate a state change sequence. Calculate the change amplitude at each time point based on the state change sequence to generate a change amplitude sequence.

[0017] Based on the change amplitude sequence, a sliding accumulation calculation is performed within a fixed-length time window to sum the change amplitudes within the window and generate a change density sequence.

[0018] Based on the change density sequence, a continuous change sequence is generated through an exponential mapping relationship, so that the continuous value decreases when the change density increases;

[0019] For continuously changing sequences, threshold comparison and adjacent time difference calculation are performed to filter time points where the continuous value is lower than the continuity threshold and the adjacent difference is greater than the abrupt change threshold, and a candidate breakpoint set is generated.

[0020] For the candidate fracture point set, the direction of change of the operating state is calculated for the difference results of the adjacent operating state vectors corresponding to each candidate fracture point, and the direction change is obtained by calculating the direction change through the vector angle. Time points with the direction change greater than the direction change threshold are selected to generate a fracture candidate enhancement set.

[0021] For each time point in the fracture candidate enhancement set, an interval expansion process is performed to expand each time point into a time interval of fixed length. Overlapping time intervals are merged to generate a set of trajectory fracture locations.

[0022] Optionally, the policy space construction module includes:

[0023] Obtain the set of trajectory break locations and the running trajectory; extract the corresponding break segments from the running trajectory based on the set of trajectory break locations, and generate a set of break segments.

[0024] Based on the set of fracture segments, the sequence of operational state vectors corresponding to each fracture segment is extracted from the operational trajectory to generate a set of fracture sub-trajectories.

[0025] For the set of fractured sub-trajectories, a set of trajectory reconstruction strategies is generated, and the set of trajectory reconstruction strategies is combined to form a trajectory reconstruction strategy space;

[0026] Based on the trajectory reconstruction strategy space, state generation processing is performed on each fracture sub-trajectory in the fracture sub-trajectory set to generate a candidate reconstruction state sequence.

[0027] Perform adjacent time state difference constraint processing on the candidate reconstruction state sequence to generate continuous candidate reconstruction trajectories;

[0028] The continuous candidate reconstructed trajectories are aggregated and the continuous candidate reconstructed trajectories corresponding to each trajectory reconstruction strategy are set together to generate a candidate reconstructed trajectory set.

[0029] Optionally, the joint characterization module includes:

[0030] Obtain the running trajectory and the set of candidate reconstructed trajectories, align the running trajectory and the set of candidate reconstructed trajectories according to the time index, and generate aligned trajectory data;

[0031] Perform adjacent state difference processing on the running trajectory in the aligned trajectory data to generate a running trajectory change sequence. Concatenate the running trajectory change sequence with the running state vector in time order to generate a running trajectory feature sequence.

[0032] Perform adjacent state difference processing on the candidate reconstructed trajectory set in the aligned trajectory data to generate a candidate trajectory change sequence. Concatenate the candidate trajectory change sequence with the corresponding running state vector in chronological order to generate a candidate trajectory feature set.

[0033] The running trajectory feature sequence and the candidate trajectory feature set are concatenated according to the same time index to generate a joint feature sequence set;

[0034] The difference between the running state vectors of the running trajectories and the candidate reconstructed trajectories in the joint feature sequence set under the same time index is calculated to generate a trajectory difference sequence.

[0035] Using a set of joint feature sequences and trajectory difference sequences as input, a state-action correspondence is constructed to generate joint representation data, where the state is the joint feature sequence and the action is the trajectory reconstruction strategy.

[0036] The joint characterization data is input into the improved IQL to generate trajectory feature representations.

[0037] Optionally, the improvements in the improved IQL include:

[0038] In the input layer, the joint feature sequence in the joint representation data is used as the state input, and the trajectory reconstruction strategy in the trajectory reconstruction strategy space is used as the action input to construct a trajectory-level state-action input structure.

[0039] In the representation layer, the joint feature sequence is processed by state encoding in chronological order, and the state information of different time indices is integrated to form a trajectory-level state representation.

[0040] At the structural layer, the connection relationship between time indices is built inside the trajectory-level state representation. Association weights are introduced to state nodes with different time indices, and an information propagation structure is formed between each state node. The state information is weighted and fused through the association weights to generate a trajectory-level coupled state representation.

[0041] In the improvement of the value assessment layer, a state-action pair is formed by trajectory-level coupled state representation and trajectory reconstruction strategy. Value assessment is performed on the trajectory reconstruction strategy, and the value assessment object is expanded from single-step action to trajectory strategy.

[0042] At the mechanism level, trajectory difference sequences from joint representation data are introduced to perform difference correction on the value assessment results, and a correlation is established between the value assessment results and the degree of difference between the running trajectory and the candidate reconstructed trajectory.

[0043] At the strategy output layer, a numerical comparison is performed on the corrected value assessment results, and the trajectory reconstruction strategy with the largest value is selected as the optimal trajectory reconstruction strategy, thereby achieving the optimal selection of the trajectory reconstruction strategy space.

[0044] Optionally, the constraint propagation module includes:

[0045] Obtain a set of candidate reconstructed trajectories, extract the running state vector sequence corresponding to each candidate reconstructed trajectory from the set of candidate reconstructed trajectories, and generate a set of candidate trajectory state sequences.

[0046] Perform adjacent state difference processing on each candidate trajectory state sequence in the candidate trajectory state sequence set to obtain the control action sequence and generate the control action sequence set.

[0047] For the set of control action sequences, pairwise difference calculations are performed on the control actions corresponding to each time index within the same candidate trajectory, and the correlation weights between control actions are generated based on the difference magnitude, forming a path correlation weight set;

[0048] A path internal constraint propagation matrix is ​​constructed based on the path association weight set. The constraint propagation matrix is ​​then normalized to generate a constraint propagation weight matrix.

[0049] The constraint propagation weight matrix is ​​used to perform weighted fusion processing on each control action in the control action sequence set to generate a propagated and updated control action sequence set.

[0050] Based on the propagated and updated set of control action sequences, an accumulation and reconstruction process is performed on the set of candidate trajectory state sequences to generate a set of constraint propagation updated trajectories.

[0051] The set of trajectory updates for constraint propagation is processed into a set to generate a set of candidate reconstructed trajectories with consistent constraints.

[0052] Optionally, the strategy optimization module includes:

[0053] Read the joint feature sequence from the joint representation data, read each trajectory reconstruction strategy from the trajectory reconstruction strategy space, establish a correspondence between each joint feature sequence and the corresponding trajectory reconstruction strategy according to the time index, and generate a set of state-action pairs;

[0054] The joint feature sequence in the state-action pair set is calculated by performing time-by-time cumulative calculation in chronological order to obtain the trajectory-level state representation set;

[0055] The trajectory-level state representation set is paired with the corresponding trajectory reconstruction strategy one by one to generate a trajectory-level state-action pair set. The trajectory-level state-action pair set is then input into the improved IQL to obtain the initial value assessment result set.

[0056] The trajectory difference sequence is read from the joint characterization data. The difference between each time point in the trajectory difference sequence is accumulated and used as the correction amount. Subtraction correction is performed on each value result in the initial value assessment result set to generate the corrected value assessment result set.

[0057] Compare the values ​​in the modified value assessment result set, select the trajectory-level state-action pair with the largest value, and generate the optimal state-action pair set.

[0058] Extract the corresponding trajectory reconstruction strategy from the set of optimal state-action pairs, and generate the optimal trajectory reconstruction strategy.

[0059] Optionally, the control execution module includes:

[0060] Obtain the optimal trajectory reconstruction strategy and the constraint-consistent candidate reconstruction trajectories, and extract the running state vectors arranged in time order from the constraint-consistent candidate reconstruction trajectories to form the target running trajectory;

[0061] Perform differential operations on the running state vectors of adjacent times in the target running trajectory, calculate the state change at each time point, and obtain the control action sequence;

[0062] For each control action in the control action sequence, the control action is decomposed according to the component dimension of the running state vector, and the overall control action is decomposed into multiple physical quantity components to form a fractional control action sequence.

[0063] Perform upper and lower limit comparison operations on each component in the fractional control action sequence, and truncate the components that exceed the range to the corresponding boundary values ​​to obtain the constrained control action sequence;

[0064] The constraint control action sequence is smoothed point by point along the time axis. A smooth control action sequence is generated by performing a weighted average calculation on the control actions at adjacent time points.

[0065] The smooth control action sequence is mapped into control signals in chronological order, and the control actions at each time point are encoded and converted to form a set of control instructions.

[0066] The set of control commands is sent to the power transformer control terminal to drive the power transformer to perform the corresponding adjustment operation and output the intelligent control result of the power transformer.

[0067] The beneficial effects of this invention are:

[0068] This invention addresses the problems of existing technologies that rely solely on single-moment states for decision-making, ignore the overall structure of the trajectory, and lack modeling of abnormal evolution processes. It constructs a deep reinforcement learning-based control framework centered on the trajectory, generating trajectories by processing multi-source SCADA data and further identifying trajectory breakpoints. Based on this, a trajectory reconstruction strategy space is built, transforming the control problem from traditional instantaneous control to a holistic decision-making problem oriented towards the trajectory evolution process. This fundamentally improves the completeness and expressive power of control modeling. Furthermore, a joint representation module unifies the modeling of the trajectories and candidate reconstructed trajectories, introducing trajectory difference sequences to quantify the degree of deviation between them. This enables the model not only to perceive the current state of the system but also to evaluate the rationality of trajectory evolution under different strategies, thereby significantly enhancing the targeted nature of control decisions.

[0069] This invention introduces a path-internal self-constraint propagation mechanism into an improved deep reinforcement learning model. It correlates and models control actions at different time indices along the trajectory, and constructs a constraint propagation weight matrix to achieve coordinated updates of control actions in the time dimension. This ensures that control behaviors at each time point are no longer isolated but form a control structure with overall consistency. Furthermore, by introducing a trajectory difference sequence for correction during value evaluation, strategy optimization not only depends on state value but also reflects the degree of deviation between candidate trajectories and actual operating trajectories, effectively suppressing unreasonable strategies with large deviations. Finally, by back-reasoning the optimal trajectory reconstruction strategy, control instructions are generated, achieving refined control of power transformers. Compared to existing technologies, this invention significantly improves global consistency, anti-disturbance capability, and decision stability, effectively enhancing the reliability and intelligence level of power transformer operation control. Attached Figure Description

[0070] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0071] Figure 1 This is a flowchart of an intelligent control system for power transformers based on deep reinforcement learning proposed in this invention.

[0072] Figure 2 This is a schematic diagram of the path internal self-constraint propagation mechanism of a power transformer intelligent control system based on deep reinforcement learning proposed in this invention.

[0073] Figure 3 This is a schematic diagram of an improved IQL structure for a power transformer intelligent control system based on deep reinforcement learning proposed in this invention. Detailed Implementation

[0074] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0075] refer to Figures 1-3 A power transformer intelligent control system based on deep reinforcement learning includes:

[0076] The data acquisition and trajectory generation module is used to acquire SCADA multi-source operation data of power transformers and preprocess it to generate operation trajectories;

[0077] The trajectory breakage identification module is used to identify trajectory topological breaks using the running trajectory, extract abnormal change segments, and form a set of trajectory breakage locations.

[0078] The strategy space construction module is used to construct a trajectory reconstruction strategy space based on the set of trajectory breakage locations, and to convert each trajectory reconstruction strategy into a candidate reconstruction trajectory to obtain a set of candidate reconstruction trajectories.

[0079] The joint characterization module is used to input the running trajectory and the candidate reconstructed trajectory set into the improved IQL, perform joint characterization on the running trajectory and the candidate reconstructed trajectory, and generate trajectory feature representation;

[0080] The constraint propagation module is used to establish a path-internal self-constraint propagation relationship for the control actions in the candidate reconstruction trajectory within the improved IQL, and to perform coordinated adjustments on each control action based on the propagation relationship to obtain a set of constraint-consistent candidate reconstruction trajectories.

[0081] The strategy optimization module is used to evaluate the value of each trajectory reconstruction strategy in the improved IQL based on the constraint-consistent candidate reconstruction trajectory set, using the trajectory reconstruction strategy space as the action space, and to complete the strategy optimization and output the optimal trajectory reconstruction strategy.

[0082] The control execution module is used to deduce the corresponding control action sequence based on the optimal trajectory reconstruction strategy, generate power transformer control instructions, and obtain intelligent control results for the power transformer.

[0083] In this embodiment, the preprocessing specifically includes data synchronization and alignment, abnormal data removal, missing data completion, multi-source data normalization, temporal resampling, state feature construction, and continuous trajectory stitching.

[0084] In this embodiment, the trajectory breakage identification module includes:

[0085] The operating trajectory is obtained, which consists of operating state vectors arranged in chronological order. Each operating state vector includes voltage, current, temperature and load parameters.

[0086] Perform differential calculation on adjacent running state vectors in the running trajectory to generate a state change sequence. Calculate the change amplitude at each time point based on the state change sequence to generate a change amplitude sequence.

[0087] Based on the change amplitude sequence, a sliding accumulation calculation is performed within a fixed-length time window to sum the change amplitudes within the window and generate a change density sequence.

[0088] Based on the change density sequence, a continuous change sequence is generated through an exponential mapping relationship, so that the continuous value decreases when the change density increases;

[0089] For continuously changing sequences, threshold comparison and adjacent time difference calculation are performed to filter time points where the continuous value is lower than the continuity threshold and the adjacent difference is greater than the abrupt change threshold, and a candidate breakpoint set is generated.

[0090] For the candidate fracture point set, the direction of change of the operating state is calculated for the difference results of the adjacent operating state vectors corresponding to each candidate fracture point, and the direction change is obtained by calculating the direction change through the vector angle. Time points with the direction change greater than the direction change threshold are selected to generate a fracture candidate enhancement set.

[0091] For each time point in the fracture candidate enhancement set, an interval expansion process is performed to expand each time point into a time interval of fixed length. Overlapping time intervals are merged to generate a set of trajectory fracture locations.

[0092] In this embodiment, the strategy space construction module includes:

[0093] Obtain the set of trajectory break locations and the running trajectory; extract the corresponding break segments from the running trajectory based on the set of trajectory break locations, and generate a set of break segments.

[0094] Based on the set of fracture segments, the sequence of operational state vectors corresponding to each fracture segment is extracted from the operational trajectory to generate a set of fracture sub-trajectories.

[0095] For the set of fractured sub-trajectories, a set of trajectory reconstruction strategies is generated, and the set of trajectory reconstruction strategies is combined to form a trajectory reconstruction strategy space;

[0096] Based on the trajectory reconstruction strategy space, state generation processing is performed on each fracture sub-trajectory in the fracture sub-trajectory set to generate a candidate reconstruction state sequence.

[0097] Perform adjacent time state difference constraint processing on the candidate reconstruction state sequence to generate continuous candidate reconstruction trajectories;

[0098] The continuous candidate reconstructed trajectories are aggregated and the continuous candidate reconstructed trajectories corresponding to each trajectory reconstruction strategy are set together to generate a candidate reconstructed trajectory set.

[0099] In this embodiment, the joint characterization module includes:

[0100] Obtain the running trajectory and the set of candidate reconstructed trajectories, align the running trajectory and the set of candidate reconstructed trajectories according to the time index, and generate aligned trajectory data;

[0101] Perform adjacent state difference processing on the running trajectory in the aligned trajectory data to generate a running trajectory change sequence. Concatenate the running trajectory change sequence with the running state vector in time order to generate a running trajectory feature sequence.

[0102] Perform adjacent state difference processing on the candidate reconstructed trajectory set in the aligned trajectory data to generate a candidate trajectory change sequence. Concatenate the candidate trajectory change sequence with the corresponding running state vector in chronological order to generate a candidate trajectory feature set.

[0103] The running trajectory feature sequence and the candidate trajectory feature set are concatenated according to the same time index to generate a joint feature sequence set;

[0104] The difference between the running state vectors of the running trajectories and the candidate reconstructed trajectories in the joint feature sequence set under the same time index is calculated to generate a trajectory difference sequence.

[0105] The generation of trajectory difference sequences specifically includes:

[0106] The system reads the running state vector sequences corresponding to the running trajectory and the running state vector sequences corresponding to the candidate reconstructed trajectories from the joint feature sequence set, and aligns them according to a unified time index to form a state alignment sequence. For each time index position in the state alignment sequence, it performs component-wise difference calculation on the running state vector of the running trajectory and the running state vector of the corresponding candidate reconstructed trajectory to obtain the state difference vector corresponding to each time point, generating a difference vector sequence. It performs amplitude calculation on each difference vector in the difference vector sequence, and synthesizes each component difference according to the vector norm to obtain the difference amplitude corresponding to each time point, generating a difference amplitude sequence. It arranges the difference amplitude sequence in chronological order, and performs continuity preservation processing on the difference amplitude of adjacent time points to keep the difference changes continuously distributed on the time axis, generating a continuous difference amplitude sequence. It binds the continuous difference amplitude sequence with the corresponding time index to form a difference sequence data structure with time identifier, generating a trajectory difference sequence.

[0107] Using a set of joint feature sequences and trajectory difference sequences as input, a state-action correspondence is constructed to generate joint representation data, where the state is the joint feature sequence and the action is the trajectory reconstruction strategy.

[0108] The generation of joint characterization data specifically includes:

[0109] The system reads the joint feature sequences corresponding to each candidate reconstructed trajectory from the joint feature sequence set, indexes and groups them according to the trajectory reconstruction strategy, and categorizes the joint feature sequences corresponding to each trajectory reconstruction strategy to generate a strategy-associated joint feature sequence set. The joint feature sequences in the strategy-associated joint feature sequence set are arranged in chronological order, and the multi-time index state vectors corresponding to the same trajectory reconstruction strategy are integrated to generate a trajectory-level state representation set. The system reads the difference amplitude data corresponding to each joint feature sequence from the trajectory difference sequence, and matches it one by one with the trajectory-level state representation according to the time index, embedding the difference amplitude into the corresponding time position to generate a difference-enhanced state representation set. The system reads each trajectory reconstruction strategy from the trajectory reconstruction strategy space and binds each trajectory reconstruction strategy with its corresponding difference-enhanced state representation to form a correspondence between state and trajectory reconstruction strategy, generating a state-action correspondence set. The system encapsulates the state-action correspondence set in a structured manner, using the difference-enhanced state representation as a state data item and the trajectory reconstruction strategy as an action data item to generate a state-action pair set. The system performs set-based processing on the state-action pair set to form joint representation data.

[0110] The joint characterization data is input into the improved IQL to generate trajectory feature representations.

[0111] In this embodiment, the improvements to the improved IQL include:

[0112] In the input layer, the joint feature sequence in the joint representation data is used as the state input, and the trajectory reconstruction strategy in the trajectory reconstruction strategy space is used as the action input to construct a trajectory-level state-action input structure.

[0113] In the representation layer, the joint feature sequence is processed by state encoding in chronological order, and the state information of different time indices is integrated to form a trajectory-level state representation.

[0114] At the structural layer, the connection relationship between time indices is built inside the trajectory-level state representation. Association weights are introduced to state nodes with different time indices, and an information propagation structure is formed between each state node. The state information is weighted and fused through the association weights to generate a trajectory-level coupled state representation.

[0115] The generation of trajectory-level coupled state representations specifically includes:

[0116] The system reads the state vectors corresponding to each time index from the trajectory-level state representation and arranges them in chronological order to form a state node sequence. For each state node in the state node sequence, based on the time index, a connection relationship is established between each state node and the state nodes of its preceding and following time indices, generating an initial state connection set. Based on the initial state connection set, pairwise difference calculations are performed on the state vectors of each state node and other time index state nodes, and association weights are assigned to each state node pair according to the magnitude of the difference, generating a state association weight set. According to the state association weight set, a weighted calculation is performed on the state vectors corresponding to each state node, and the state vectors under the same time index are weighted and accumulated with the state vectors of other time indices according to the association weights, generating an intermediate coupled state sequence. The intermediate coupled state sequence is arranged in chronological order, and consistency constraint processing is performed on the coupled states of adjacent time indices to keep the state changes under each time index continuous, generating a continuous coupled state sequence. The continuous coupled state sequence is encapsulated into a set to form a trajectory-level coupled state representation.

[0117] In the improvement of the value assessment layer, a state-action pair is formed by trajectory-level coupled state representation and trajectory reconstruction strategy. Value assessment is performed on the trajectory reconstruction strategy, and the value assessment object is expanded from single-step action to trajectory strategy.

[0118] Valuation specifically includes:

[0119] The process involves: reading the state vector sequence corresponding to each trajectory from the trajectory-level coupled state representation; performing index matching according to the trajectory reconstruction strategy; associating the trajectory-level coupled state representation corresponding to each trajectory reconstruction strategy to generate a trajectory-level state-action pair set; performing time-by-time cumulative calculation on each trajectory-level coupled state representation in the trajectory-level state-action pair set; integrating the state vectors under multiple time indices to generate a trajectory-level state aggregation vector set; combining the trajectory-level state aggregation vector set with the corresponding trajectory reconstruction strategy; performing value calculation on each state-action pair to generate an initial value assessment result set; grouping the initial value assessment result set according to the trajectory reconstruction strategy; summarizing the values ​​of multiple time segments corresponding to the same trajectory reconstruction strategy to generate a trajectory-level value sequence set; arranging the value sequences in the trajectory-level value sequence set in chronological order; performing consistency constraint processing on the values ​​of adjacent time points to generate a continuous trajectory value sequence set; and performing overall numerical synthesis on the continuous trajectory value sequence set, accumulating the values ​​under each time index to generate the value assessment result corresponding to the trajectory reconstruction strategy.

[0120] At the mechanism level, trajectory difference sequences from joint representation data are introduced to perform difference correction on the value assessment results, and a correlation is established between the value assessment results and the degree of difference between the running trajectory and the candidate reconstructed trajectory.

[0121] Establishing a relationship specifically includes:

[0122] The process involves: reading the difference magnitude corresponding to each time index from the trajectory difference sequence and arranging them chronologically to generate a difference magnitude sequence; matching the difference magnitude sequence with the time index corresponding to each trajectory reconstruction strategy in the value assessment result set, and mapping the difference magnitude at each time point to the corresponding value assessment result to generate a difference-value matching sequence; performing cumulative calculation on each difference magnitude in the difference-value matching sequence to obtain the total trajectory-level difference, using the total trajectory-level difference as a correction amount to generate a difference correction amount set; applying the difference correction amount set to each trajectory reconstruction strategy in the value assessment result set, performing subtraction correction on each value assessment result to generate a difference correction value result set; constraining and adjusting each trajectory reconstruction strategy in the difference correction value result set according to the size of the difference magnitude, reducing the value result corresponding to the trajectory with a larger difference magnitude, and generating a difference-related value sequence; and performing set processing on the difference-related value sequence to form the correlation between trajectory differences and value assessment results.

[0123] At the strategy output layer, a numerical comparison is performed on the corrected value assessment results, and the trajectory reconstruction strategy with the largest value is selected as the optimal trajectory reconstruction strategy, thereby achieving the optimal selection of the trajectory reconstruction strategy space.

[0124] The improved IQL reconstructs the input structure, state representation method, internal structural relationships, and value assessment mechanism compared to the traditional IQL. Traditional IQL typically uses the state at a single point in time as input and a single action as the decision-making object, with a one-to-one correspondence between state and action, lacking the ability to depict the overall evolution of the time series. In this implementation, the joint feature sequence is constructed as a trajectory-level state input, and the trajectory reconstruction strategy is used as the action input, expanding the correspondence between state and action from a single step to a trajectory-level correspondence, thereby achieving holistic modeling of the entire system operation process. Regarding state representation, the traditional... In IQL, states at different time points are independent of each other, and no information connection is established across time indices. This implementation constructs a connection relationship between time indices within the trajectory-level state representation and introduces association weights to form an information propagation structure between state nodes under different time indices, thereby generating a trajectory-level coupled state representation. In this way, state information can be fused in the time dimension, avoiding decision bias caused by isolated states at a single moment. Regarding the value assessment mechanism, traditional IQL usually calculates value based on single-step state-action pairs, which cannot reflect the comprehensive effect of the strategy over the entire time interval. This implementation incorporates value assessment... The evaluation object is expanded from a single-step action to a trajectory reconstruction strategy. By accumulating and integrating state information under multiple time indices, a trajectory-level value evaluation result is generated, enabling the value evaluation to reflect the overall performance of the strategy on the complete trajectory. At the mechanism level, traditional IQL's value evaluation relies solely on state features, without considering the degree of deviation between candidate trajectories and the actual running trajectory. This implementation introduces a trajectory difference sequence, using trajectory differences as a correction factor in the value evaluation process. By performing difference correction on the value results, candidate trajectories with large deviations are suppressed in the value evaluation, thereby achieving difference constraints on strategy selection. In the strategy selection stage, traditional IQL directly selects the maximum value based on the value function, while this implementation performs strategy selection based on the value evaluation result after difference correction. This ensures that the final output trajectory reconstruction strategy not only has a high value evaluation result but also maintains a high degree of consistency with the actual running trajectory. This implementation expands the state from a single point to a trajectory-level structure, introduces a time-dimensional information propagation mechanism, and integrates trajectory difference constraints in the value evaluation process, achieving a structural improvement over traditional IQL. This transforms the model from local decision-making to global trajectory decision-making, thereby improving the stability, consistency, and adaptability of the control strategy to complex operating environments.

[0125] In this embodiment, the constraint propagation module includes:

[0126] Obtain a set of candidate reconstructed trajectories, extract the running state vector sequence corresponding to each candidate reconstructed trajectory from the set of candidate reconstructed trajectories, and generate a set of candidate trajectory state sequences.

[0127] Perform adjacent state difference processing on each candidate trajectory state sequence in the candidate trajectory state sequence set to obtain the control action sequence and generate the control action sequence set.

[0128] For the set of control action sequences, pairwise difference calculations are performed on the control actions corresponding to each time index within the same candidate trajectory, and the correlation weights between control actions are generated based on the difference magnitude, forming a path correlation weight set;

[0129] A path internal constraint propagation matrix is ​​constructed based on the path association weight set. The constraint propagation matrix is ​​then normalized to generate a constraint propagation weight matrix.

[0130] The generation of the constraint propagation weight matrix specifically includes:

[0131] The algorithm reads the control action association weight data corresponding to each candidate reconstructed trajectory from the path association weight set, sorts the association relationships between each control action according to the time index, and generates an ordered association weight sequence. For the ordered association weight sequence, the control action association weights between each time index are filled into the corresponding positions using the time index as the row and column identifiers to construct an initial path association matrix. The algorithm performs row-by-row summation on each row element in the initial path association matrix to generate a row weight total sequence, which is used as a normalization factor set. Each row element in the initial path association matrix is ​​divided by the corresponding row weight total, and the elements in the matrix are scaled proportionally to generate a row normalization matrix, so that the sum of the weights in each row is equal to a preset constant. The row normalization matrix is ​​subjected to numerical stability constraint processing, raising the weight elements below the preset lower limit to the minimum threshold and truncating the weight elements above the preset upper limit to generate a stable constraint matrix. The stable constraint matrix is ​​encapsulated into a set to form a constraint propagation weight matrix. By mapping the association weights between control actions to the path association matrix and further normalizing them to form a constraint propagation weight matrix, a structured expression of the association relationships between control actions is realized.

[0132] The constraint propagation weight matrix is ​​used to perform weighted fusion processing on each control action in the control action sequence set to generate a propagated and updated control action sequence set.

[0133] The generation of the set of regulatory action sequences specifically includes:

[0134] The system reads the weight matrix data corresponding to each candidate reconstruction trajectory from the constraint propagation weight matrix, and reads the control action sequence corresponding to each candidate reconstruction trajectory from the control action sequence set. The control action sequence is aligned with the constraint propagation weight matrix according to the time index to generate a weight-action alignment sequence. For each time index position in the weight-action alignment sequence, the weight vector of the corresponding time index and the control action vector under all time indices are read. The weight vector and the control action vector are multiplied component-by-component, and the product results are accumulated to generate the weighted control action vector corresponding to that time index. The weighted calculation process is repeated for all time index positions. The weighted control action vectors generated under each time index are arranged in chronological order to generate a propagation update control action sequence. Amplitude constraint processing is performed on each control action vector in the propagation update control action sequence. Components exceeding the preset upper limit are truncated, and components below the preset lower limit are boosted to generate a constraint propagation control action sequence. The constraint propagation control action sequence is grouped according to the candidate reconstruction trajectory, and the control action sequences corresponding to each candidate reconstruction trajectory are aggregated to generate a propagation update control action sequence set.

[0135] Based on the propagated and updated set of control action sequences, an accumulation and reconstruction process is performed on the set of candidate trajectory state sequences to generate a set of constraint propagation updated trajectories.

[0136] The generation of the constraint propagation update trajectory set specifically includes:

[0137] The system reads the control action sequences corresponding to each candidate reconstructed trajectory from the set of updated control action sequences and the initial running state vectors corresponding to each candidate reconstructed trajectory from the set of candidate trajectory state sequences. It aligns the control action sequences with the initial running state vectors according to time indices to generate a state-action alignment sequence. For the state-action alignment sequence, starting from the initial running state vector, it progressively superimposes the control action vectors under the corresponding time indices in chronological order, performing state update calculations for each time index to generate a time-by-time updated state sequence. The time-by-time updated state sequence is arranged in chronological order, and continuous constraint processing is performed on the running state vectors under adjacent time indices to ensure that state changes meet preset continuous change conditions, generating a continuously updated state sequence. The continuously updated state sequence is grouped according to the candidate reconstructed trajectories, and the updated state sequence corresponding to each candidate reconstructed trajectory is encapsulated to generate a set of propagation updated trajectories. State consistency checks are performed on each updated trajectory in the propagation updated trajectory set, and states that do not meet consistency constraints are adjusted to ensure that each updated trajectory meets preset running constraint conditions, generating a set of constrained propagation updated trajectories.

[0138] The set of trajectory updates for constraint propagation is processed into a set to generate a set of candidate reconstructed trajectories with consistent constraints.

[0139] In this embodiment, the strategy optimization module includes:

[0140] Read the joint feature sequence from the joint representation data, read each trajectory reconstruction strategy from the trajectory reconstruction strategy space, establish a correspondence between each joint feature sequence and the corresponding trajectory reconstruction strategy according to the time index, and generate a set of state-action pairs;

[0141] The joint feature sequence in the state-action pair set is calculated by performing time-by-time cumulative calculation in chronological order to obtain the trajectory-level state representation set;

[0142] The trajectory-level state representation set is paired with the corresponding trajectory reconstruction strategy one by one to generate a trajectory-level state-action pair set. The trajectory-level state-action pair set is then input into the improved IQL to obtain the initial value assessment result set.

[0143] The trajectory difference sequence is read from the joint characterization data. The difference between each time point in the trajectory difference sequence is accumulated and used as the correction amount. Subtraction correction is performed on each value result in the initial value assessment result set to generate the corrected value assessment result set.

[0144] The trajectory difference sequence is used to characterize the degree of state deviation between the running trajectory and the candidate reconstructed trajectory at the same time index. Since the difference amplitude at a single time point is easily affected by local fluctuations and cannot reflect the overall trajectory deviation, the difference amplitudes at each time point in the trajectory difference sequence are accumulated to obtain the total trajectory-level difference. The total trajectory-level difference can reflect the overall deviation of the candidate reconstructed trajectory relative to the running trajectory over the entire time interval. The total trajectory-level difference is introduced as a correction factor into the value evaluation process, and the initial value evaluation result is subtracted to reduce the value evaluation result of the trajectory with a larger difference amplitude. In this way, when selecting a strategy, not only the running characteristics reflected by the trajectory-level state representation are considered, but also the degree of deviation between the candidate trajectory and the actual running trajectory are considered simultaneously, thereby achieving difference constraints on the trajectory reconstruction strategy. Through this mechanism, a correlation is established between the value evaluation result and the degree of trajectory deviation, avoiding the selection of candidate reconstruction strategies that deviate significantly from the running trajectory, and improving the stability and reliability of the strategy optimization results.

[0145] Compare the values ​​in the modified value assessment result set, select the trajectory-level state-action pair with the largest value, and generate the optimal state-action pair set.

[0146] Extract the corresponding trajectory reconstruction strategy from the set of optimal state-action pairs, and generate the optimal trajectory reconstruction strategy.

[0147] In this embodiment, the control execution module includes:

[0148] Obtain the optimal trajectory reconstruction strategy and the constraint-consistent candidate reconstruction trajectories, and extract the running state vectors arranged in time order from the constraint-consistent candidate reconstruction trajectories to form the target running trajectory;

[0149] Perform differential operations on the running state vectors of adjacent times in the target running trajectory, calculate the state change at each time point, and obtain the control action sequence;

[0150] For each control action in the control action sequence, the control action is decomposed according to the component dimension of the running state vector, and the overall control action is decomposed into multiple physical quantity components to form a fractional control action sequence.

[0151] Perform upper and lower limit comparison operations on each component in the fractional control action sequence, and truncate the components that exceed the range to the corresponding boundary values ​​to obtain the constrained control action sequence;

[0152] The constraint control action sequence is smoothed point by point along the time axis. A smooth control action sequence is generated by performing a weighted average calculation on the control actions at adjacent time points.

[0153] The smooth control action sequence is mapped into control signals in chronological order, and the control actions at each time point are encoded and converted to form a set of control instructions.

[0154] The set of control commands is sent to the power transformer control terminal to drive the power transformer to perform the corresponding adjustment operation and output the intelligent control result of the power transformer.

[0155] Example 1: To verify the feasibility of this invention in practice, it was applied to the operation and control scenario of the main transformer in a coastal urban power grid. The power grid load in this area exhibits obvious diurnal fluctuations and is affected by temperature changes and the periodicity of industrial electricity consumption. The transformer operates under complex and fluctuating conditions for a long time. Existing control methods mainly rely on dispatchers to make time-sharing adjustments based on historical experience, or on traditional control algorithms to make adjustments based on instantaneous monitoring data. It is difficult to effectively characterize the load change trend and abnormal evolution process, resulting in a lag in control response when the load fluctuates rapidly or local anomalies spread. This can easily lead to problems such as unstable voltage fluctuations and unbalanced load distribution. This invention is deployed in such scenarios. By introducing a trajectory-level deep reinforcement learning mechanism, the entire transformer operation process is modeled and optimized, thereby solving the problems of strong locality and insufficient global consistency in control decisions in existing technologies.

[0156] In practical applications, the system first connects to the power grid SCADA system to continuously collect operating data of the main transformer in the area at different time periods, including multi-source information such as voltage, current, temperature, and load changes. The collected data undergoes time synchronization alignment, anomaly removal, missing data completion, and normalization to generate a continuous operating trajectory. Based on this, the system performs trajectory topology break identification on the operating trajectory. By analyzing the continuity and direction of changes in operating status, it locates time segments with abrupt change characteristics, thereby identifying potential abnormal evolution locations. For these break segments, a trajectory reconstruction strategy space is further constructed, mapping different control strategies as candidate reconstruction trajectories, enabling the system to select from multiple potential evolution paths.

[0157] The running trajectory and candidate reconstructed trajectories are jointly represented. Features of the two types of trajectories are aligned under the same time index, and a trajectory difference sequence is generated through difference calculation. This allows the model to perceive the degree of deviation between the candidate trajectory and the actual running trajectory. Based on this, the joint feature sequence is used as the state input, and the trajectory reconstruction strategy is used as the action input to construct a trajectory-level state-action correspondence, which is then input into the improved reinforcement learning model. Inside the model, by constructing connections between time indices, the state information at different time points can influence each other, thus forming a trajectory-level coupled state representation. Simultaneously, a path-internal self-constraint propagation mechanism is introduced at the control action level. By constructing a constraint propagation weight matrix, the control actions at each time point are weighted and fused during propagation, avoiding discontinuous effects of local control behaviors on the overall trajectory.

[0158] During the strategy optimization process, a trajectory difference sequence is introduced to correct the value assessment results, which suppresses candidate strategies that deviate significantly from the actual operating trajectory in the value assessment. This allows for the priority selection of control paths that align with the actual operating trend. Finally, based on the optimized trajectory reconstruction strategy, the corresponding control action sequence is derived and converted into control commands, which are then sent to the transformer control terminal to achieve dynamic adjustment of voltage and load distribution. Throughout the process, the control behavior no longer relies on the state judgment at a single moment but makes global decisions based on the complete operating trajectory, thereby significantly improving the stability and continuity of the control.

[0159] Based on the operational time records, this method can formulate a response strategy at the initial stage of load change trends over multiple continuous operating cycles. The generation and execution of control commands remain continuous, avoiding the frequent adjustments caused by local fluctuations seen in traditional methods. From a spatial distribution perspective, the load distribution on each side of the transformer is more balanced, without local overload or voltage fluctuation propagation issues. In terms of control process performance, the control actions exhibit smooth transition characteristics over time, with consistent adjustment behavior across different time points, avoiding instability caused by independent decision-making. Furthermore, by introducing and constraining trajectory differences, the system can proactively avoid deviations from the path when facing abnormal fluctuations, resulting in control results that more closely match actual operational needs.

[0160] Table 1 Performance Comparison Data of Intelligent Control Methods for Power Transformers

[0161] Indicator Item Traditional rule control methods Conventional reinforcement learning methods Method of the present invention Voltage fluctuation amplitude (pu) 0.042 0.031 0.018 Load distribution deviation rate (%) 7.6 5.2 2.9 Control response delay (s) 4.8 3.6 2.1 Rate of change in control actions (times / minute) 5.7 4.3 2.5 Abnormal disturbance recovery time (s) 18.5 13.2 7.9 Control trajectory continuity error 0.036 0.025 0.014 Policy convergence and stabilization time (min) 26.4 19.7 11.8

[0162] As shown in Table 1, the voltage fluctuation amplitude index is 0.042 for the traditional rule-based control method, 0.031 for the conventional reinforcement learning method, and further reduced to 0.018 for the method of this invention. This indicates that the present invention has a significant advantage in suppressing voltage fluctuations. The results show that by introducing trajectory-level state representation and trajectory difference constraint mechanism, the control strategy can respond to voltage change trends in advance, thereby reducing fluctuation amplitude. In terms of load distribution deviation rate, the traditional method is 7.6%, the conventional reinforcement learning method is 5.2%, and the method of this invention reduces it to 2.9%. This improvement reflects that the present invention, by constructing candidate reconstruction trajectories and performing strategy screening, makes the load distribution more balanced in different time periods, avoiding the deviation problem caused by local load concentration.

[0163] Regarding the control response delay, the method of this invention reduces the delay from 4.8 seconds in the traditional method to 2.1 seconds, and further reduces it compared to conventional reinforcement learning methods. This result indicates that, through trajectory breakage identification and policy space construction, this invention can form a control strategy at the initial stage of changes in the operating state, thereby reducing response lag. In terms of the rate of change of control actions, the traditional method is 5.7 times per minute, the conventional reinforcement learning method is 4.3 times per minute, while the method of this invention reduces it to 2.5 times per minute, indicating more stable control actions. This result stems from the self-constrained propagation mechanism within the path, ensuring consistency of control actions at different time points during propagation and reducing unnecessary frequent adjustments.

[0164] Regarding the recovery time from abnormal disturbances, the method of this invention reduces it to 7.9 seconds, which is more than half that of the traditional method. This indicates that the system can recover to a stable state more quickly when faced with sudden disturbances. This is mainly due to the constraint effect of the trajectory difference sequence on the strategy, which effectively suppresses candidate trajectories with large deviations during the strategy optimization process. In the continuity error index of the control trajectory, the method of this invention is 0.014, which is significantly lower than the 0.036 of the traditional method. This indicates that the control trajectory is smoother and more continuous. This result is closely related to the introduction of the constraint propagation weight matrix, which makes the control action correlated in the time dimension, thereby avoiding abrupt changes.

[0165] Regarding the policy convergence and stabilization time, the method of this invention reduces it to 11.8 minutes, a significant decrease compared to traditional reinforcement learning methods, indicating a more efficient policy optimization process. This is because trajectory-level state-action modeling reduces the ineffective exploration space, while the difference correction mechanism improves the accuracy of value assessment. In summary, the above data shows that this invention, through trajectory-level modeling, constraint propagation mechanism, and difference correction mechanism, transforms the control process from single-point decision-making to trajectory-level collaborative decision-making, achieving quantitative improvements in multiple key performance indicators, particularly in stability, response speed, and global consistency.

[0166] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A power transformer intelligent control system based on deep reinforcement learning, characterized in that, include: The data acquisition and trajectory generation module is used to acquire SCADA multi-source operation data of power transformers and preprocess it to generate operation trajectories; The trajectory breakage identification module is used to identify trajectory topological breaks using the running trajectory, extract abnormal change segments, and form a set of trajectory breakage locations. The strategy space construction module is used to construct a trajectory reconstruction strategy space based on the set of trajectory breakage locations, and to convert each trajectory reconstruction strategy into a candidate reconstruction trajectory to obtain a set of candidate reconstruction trajectories. The joint characterization module is used to input the running trajectory and the candidate reconstructed trajectory set into the improved IQL, perform joint characterization on the running trajectory and the candidate reconstructed trajectory, and generate trajectory feature representation; The constraint propagation module is used to establish a path-internal self-constraint propagation relationship for the control actions in the candidate reconstruction trajectory within the improved IQL, and to perform coordinated adjustments on each control action based on the propagation relationship to obtain a set of constraint-consistent candidate reconstruction trajectories. The strategy optimization module is used to evaluate the value of each trajectory reconstruction strategy in the improved IQL based on the constraint-consistent candidate reconstruction trajectory set, using the trajectory reconstruction strategy space as the action space, and to complete the strategy optimization and output the optimal trajectory reconstruction strategy. The control execution module is used to deduce the corresponding control action sequence based on the optimal trajectory reconstruction strategy, generate power transformer control instructions, and obtain intelligent control results for the power transformer.

2. The intelligent control system for power transformers based on deep reinforcement learning according to claim 1, characterized in that, The preprocessing specifically includes data synchronization and alignment, abnormal data removal, missing data completion, multi-source data normalization, temporal resampling, state feature construction, and continuous trajectory stitching.

3. The intelligent control system for power transformers based on deep reinforcement learning according to claim 1, characterized in that, The trajectory breakage identification module includes: The operating trajectory is obtained, which consists of operating state vectors arranged in chronological order. Each operating state vector includes voltage, current, temperature and load parameters. Perform differential calculation on adjacent running state vectors in the running trajectory to generate a state change sequence. Calculate the change amplitude at each time point based on the state change sequence to generate a change amplitude sequence. Based on the change amplitude sequence, a sliding accumulation calculation is performed within a fixed-length time window to sum the change amplitudes within the window and generate a change density sequence. Based on the change density sequence, a continuous change sequence is generated through an exponential mapping relationship, so that the continuous value decreases when the change density increases; For continuously changing sequences, threshold comparison and adjacent time difference calculation are performed to filter time points where the continuous value is lower than the continuity threshold and the adjacent difference is greater than the abrupt change threshold, and a candidate breakpoint set is generated. For the candidate fracture point set, the direction of change of the operating state is calculated for the difference results of the adjacent operating state vectors corresponding to each candidate fracture point, and the direction change is obtained by calculating the direction change through the vector angle. Time points with the direction change greater than the direction change threshold are selected to generate a fracture candidate enhancement set. For each time point in the fracture candidate enhancement set, an interval expansion process is performed to expand each time point into a time interval of fixed length. Overlapping time intervals are merged to generate a set of trajectory fracture locations.

4. The intelligent control system for power transformers based on deep reinforcement learning according to claim 1, characterized in that, The strategy space construction module includes: Obtain the set of trajectory break locations and the running trajectory; extract the corresponding break segments from the running trajectory based on the set of trajectory break locations, and generate a set of break segments. Based on the set of fracture segments, the sequence of operational state vectors corresponding to each fracture segment is extracted from the operational trajectory to generate a set of fracture sub-trajectories. For the set of fractured sub-trajectories, a set of trajectory reconstruction strategies is generated, and the set of trajectory reconstruction strategies is combined to form a trajectory reconstruction strategy space; Based on the trajectory reconstruction strategy space, state generation processing is performed on each fracture sub-trajectory in the fracture sub-trajectory set to generate a candidate reconstruction state sequence. Perform adjacent time state difference constraint processing on the candidate reconstruction state sequence to generate continuous candidate reconstruction trajectories; The continuous candidate reconstructed trajectories are aggregated and the continuous candidate reconstructed trajectories corresponding to each trajectory reconstruction strategy are set together to generate a candidate reconstructed trajectory set.

5. The intelligent control system for power transformers based on deep reinforcement learning according to claim 1, characterized in that, The joint characterization module includes: Obtain the running trajectory and the set of candidate reconstructed trajectories, align the running trajectory and the set of candidate reconstructed trajectories according to the time index, and generate aligned trajectory data; Perform adjacent state difference processing on the running trajectory in the aligned trajectory data to generate a running trajectory change sequence. Concatenate the running trajectory change sequence with the running state vector in time order to generate a running trajectory feature sequence. Perform adjacent state difference processing on the candidate reconstructed trajectory set in the aligned trajectory data to generate a candidate trajectory change sequence. Concatenate the candidate trajectory change sequence with the corresponding running state vector in chronological order to generate a candidate trajectory feature set. The running trajectory feature sequence and the candidate trajectory feature set are concatenated according to the same time index to generate a joint feature sequence set; The difference between the running state vectors of the running trajectories and the candidate reconstructed trajectories in the joint feature sequence set under the same time index is calculated to generate a trajectory difference sequence. Using a set of joint feature sequences and trajectory difference sequences as input, a state-action correspondence is constructed to generate joint representation data, where the state is the joint feature sequence and the action is the trajectory reconstruction strategy. The joint characterization data is input into the improved IQL to generate trajectory feature representations.

6. The intelligent control system for power transformers based on deep reinforcement learning according to claim 1, characterized in that, The improvements in the improved IQL include: In the input layer, the joint feature sequence in the joint representation data is used as the state input, and the trajectory reconstruction strategy in the trajectory reconstruction strategy space is used as the action input to construct a trajectory-level state-action input structure. In the representation layer, the joint feature sequence is processed by state encoding in chronological order, and the state information of different time indices is integrated to form a trajectory-level state representation. At the structural layer, the connection relationship between time indices is built inside the trajectory-level state representation. Association weights are introduced to state nodes with different time indices, and an information propagation structure is formed between each state node. The state information is weighted and fused through the association weights to generate a trajectory-level coupled state representation. In the improvement of the value assessment layer, a state-action pair is formed by trajectory-level coupled state representation and trajectory reconstruction strategy. Value assessment is performed on the trajectory reconstruction strategy, and the value assessment object is expanded from single-step action to trajectory strategy. At the mechanism level, trajectory difference sequences from joint representation data are introduced to perform difference correction on the value assessment results, and a correlation is established between the value assessment results and the degree of difference between the running trajectory and the candidate reconstructed trajectory. At the strategy output layer, a numerical comparison is performed on the corrected value assessment results, and the trajectory reconstruction strategy with the largest value is selected as the optimal trajectory reconstruction strategy, thereby achieving the optimal selection of the trajectory reconstruction strategy space.

7. The intelligent control system for power transformers based on deep reinforcement learning according to claim 1, characterized in that, The constraint propagation module includes: Obtain a set of candidate reconstructed trajectories, extract the running state vector sequence corresponding to each candidate reconstructed trajectory from the set of candidate reconstructed trajectories, and generate a set of candidate trajectory state sequences. Perform adjacent state difference processing on each candidate trajectory state sequence in the candidate trajectory state sequence set to obtain the control action sequence and generate the control action sequence set. For the set of control action sequences, pairwise difference calculations are performed on the control actions corresponding to each time index within the same candidate trajectory, and the correlation weights between control actions are generated based on the difference magnitude, forming a path correlation weight set; A path internal constraint propagation matrix is ​​constructed based on the path association weight set. The constraint propagation matrix is ​​then normalized to generate a constraint propagation weight matrix. The constraint propagation weight matrix is ​​used to perform weighted fusion processing on each control action in the control action sequence set to generate a propagated and updated control action sequence set. Based on the propagated and updated set of control action sequences, an accumulation and reconstruction process is performed on the set of candidate trajectory state sequences to generate a set of constraint propagation updated trajectories. The set of trajectory updates for constraint propagation is processed into a set to generate a set of candidate reconstructed trajectories with consistent constraints.

8. The intelligent control system for power transformers based on deep reinforcement learning according to claim 1, characterized in that, The strategy optimization module includes: Read the joint feature sequence from the joint representation data, read each trajectory reconstruction strategy from the trajectory reconstruction strategy space, establish a correspondence between each joint feature sequence and the corresponding trajectory reconstruction strategy according to the time index, and generate a set of state-action pairs; The joint feature sequence in the state-action pair set is calculated by performing time-by-time cumulative calculation in chronological order to obtain the trajectory-level state representation set; The trajectory-level state representation set is paired with the corresponding trajectory reconstruction strategy one by one to generate a trajectory-level state-action pair set. The trajectory-level state-action pair set is then input into the improved IQL to obtain the initial value assessment result set. The trajectory difference sequence is read from the joint characterization data. The difference between each time point in the trajectory difference sequence is accumulated and used as the correction amount. Subtraction correction is performed on each value result in the initial value assessment result set to generate the corrected value assessment result set. Compare the values ​​in the modified value assessment result set, select the trajectory-level state-action pair with the largest value, and generate the optimal state-action pair set. Extract the corresponding trajectory reconstruction strategy from the set of optimal state-action pairs, and generate the optimal trajectory reconstruction strategy.

9. The intelligent control system for power transformers based on deep reinforcement learning according to claim 1, characterized in that, The control execution module includes: Obtain the optimal trajectory reconstruction strategy and the constraint-consistent candidate reconstruction trajectories, and extract the running state vectors arranged in time order from the constraint-consistent candidate reconstruction trajectories to form the target running trajectory; Perform differential operations on the running state vectors of adjacent times in the target running trajectory, calculate the state change at each time point, and obtain the control action sequence; For each control action in the control action sequence, the control action is decomposed according to the component dimension of the running state vector, and the overall control action is decomposed into multiple physical quantity components to form a fractional control action sequence. Perform upper and lower limit comparison operations on each component in the fractional control action sequence, and truncate the components that exceed the range to the corresponding boundary values ​​to obtain the constrained control action sequence; The constraint control action sequence is smoothed point by point along the time axis. A smooth control action sequence is generated by performing a weighted average calculation on the control actions at adjacent time points. The smooth control action sequence is mapped into control signals in chronological order, and the control actions at each time point are encoded and converted to form a set of control instructions. The set of control commands is sent to the power transformer control terminal to drive the power transformer to perform the corresponding control operation and output the intelligent control result of the power transformer.

Images