A power equipment control system combined with deep reinforcement learning

Abstract

The present invention discloses a power equipment control system combined with deep reinforcement learning, specifically relating to the field of feedback voltage regulation, including a feature extraction module, a strategy determination module, a path selection module, and a feedback optimization module; the feature extraction module is used to perform structured disturbance analysis on the voltage time series data at the output end of the power equipment, and by extracting the disturbance duration and voltage slope value and comparing them with a preset threshold, a fuzzy deviation segment with feedback regulation significance is identified; the strategy determination module is used to construct a strategy mapping relationship between the disturbance feature and the regulation behavior, and to determine whether to generate a control signal to drive feedback regulation based on the regulation response score value. By constructing a regulation strategy network with disturbance features as input and stability indicators as return, and combining multi-dimensional feedback parameters to realize the identification, judgment and response control of fuzzy deviations, the problem of response inaccuracy and regulation delay of traditional feedback systems under atypical disturbances is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of feedback voltage regulation, and more specifically, to an electric power equipment control system combined with deep reinforcement learning. Background Art

[0002] In existing power systems, stable control of output voltage typically relies on a feedback regulation system based on a PID or linear controller. This system continuously monitors the output voltage and compares it with one or more predetermined target values. Once a deviation is detected, it is transmitted as a feedback signal to the control device, prompting the system to perform responsive adjustments to restore the output voltage to the desired level, thus forming a closed-loop regulation mechanism with feedback.

[0003] However, with the increasing prevalence of high-frequency disturbances in actual power grids (such as the charging and discharging of large groups of electric vehicles, the injection of distributed renewable energy, or dynamic load switching), "fuzzy deviations" with short durations (including but not limited to tens to hundreds of milliseconds) but uncertain amplitudes frequently appear in the system output voltage. These deviations neither constitute obvious steady-state offsets nor exhibit typical transient mutation characteristics, making them difficult for traditional controllers to accurately classify.

[0004] Because existing feedback systems rely on a single deviation threshold to drive regulation, they often respond incorrectly to such fuzzy disturbances. This can lead to output oscillations due to misjudgment and ineffective regulation, or delayed response that misses the opportunity for effective regulation, compromising system stability.

[0005] It can be seen that the current automatic voltage regulation system based on feedback deviation lacks response strategies and identification mechanisms when facing atypical deviation signals, which becomes a major bottleneck limiting its regulation accuracy and system robustness. Summary of the Invention

[0006] To overcome the above-mentioned shortcomings of the prior art, an embodiment of the present invention provides an electric power equipment control system combined with deep reinforcement learning. By constructing a regulation strategy network with disturbance characteristics as input and stability indicators as rewards, and combining multi-dimensional feedback parameters to realize the identification, judgment and response control of fuzzy deviations, the problems of inaccurate response and regulation delay of traditional feedback systems under atypical disturbances are solved.

[0007] To achieve the above objectives, the present invention provides the following technical solutions: a power equipment control system combined with deep reinforcement learning, comprising a feature extraction module, a strategy determination module, a path selection module, and a feedback optimization module;

[0008] The feature extraction module is used to perform structured disturbance analysis on the voltage time series data at the output of the power equipment. By extracting the disturbance duration and voltage slope value and comparing them with the preset threshold, it identifies the fuzzy deviation segments with feedback regulation significance.

[0009] The strategy decision module is used to construct a strategy mapping relationship between disturbance characteristics and regulation behaviors, and to determine whether to generate a control signal to drive feedback regulation based on the regulation response score.

[0010] The path selection module is used to determine whether to enter the regulation process based on the regulation execution flag, and select the corresponding voltage regulation device type based on the disturbance duration and voltage slope value in the voltage disturbance feature vector and the corresponding applicable threshold value. It also constructs a regulation drive parameter group based on the voltage deviation direction and sends it to the voltage regulation device to perform feedback regulation control;

[0011] The feedback optimization module is used to construct a regulation result evaluation index group based on the regulated feedback voltage data, and generate a strategy execution error value by calculating the difference with the regulation response score value, which is used to update the regulation strategy network to complete the feedback correction of the voltage regulation device.

[0012] In a preferred embodiment, the feature extraction module is used to perform real-time acquisition operations on the voltage time series data at the output end of the power equipment, obtain continuous time window segments, and then perform range calculation and local derivative operation on each segment in the continuous time window segment to output a voltage disturbance feature vector; the voltage disturbance feature vector includes the disturbance duration and the voltage slope value;

[0013] The disturbance duration and voltage slope value are compared with the preset disturbance duration threshold and voltage slope threshold respectively; if the disturbance duration is less than the disturbance duration threshold and the voltage slope value is greater than the voltage slope threshold, the paragraph is marked as a fuzzy deviation paragraph, otherwise it is marked as a non-fuzzy paragraph;

[0014] During the fuzzy deviation paragraph labeling process, each set of voltage disturbance feature vectors and their corresponding manual judgment adjustment effect labels are bound as sample pairs, and a disturbance response dataset for strategy training is output.

[0015] In a preferred embodiment, the strategy determination module is used to input the disturbance response data set into the deep reinforcement learning network to construct a state-action mapping structure during the execution of the strategy training operation, and then update the network weights based on the stability index composed of the adjusted voltage stabilization time and the residual deviation amplitude as a reward function, and output the adjustment strategy network;

[0016] In the execution strategy reasoning operation, the newly input voltage disturbance feature vector is input into the regulation strategy network to obtain the corresponding regulation response score value. If the regulation response score value is greater than the preset regulation activation threshold, the output regulation execution flag is activated, otherwise it is inactivated.

[0017] In executing the regulation execution flag output operation, the output regulation execution flag is sent as a control signal, and the current voltage disturbance feature vector and the regulation response score value are cached together for subsequent calls.

[0018] In a preferred embodiment, the path selection module is used to determine whether the adjustment execution flag is in an activated state. If it is in an activated state, the adjustment process is entered; if it is in an inactivated state, the control state of the existing voltage adjustment device in the power equipment is maintained unchanged and the operation of sending the adjustment control signal is skipped;

[0019] According to the disturbance duration and voltage slope value in the voltage disturbance characteristic vector, the disturbance duration and voltage slope value are compared with the preset duration applicable threshold and slope applicable threshold respectively. If the disturbance duration is less than the duration applicable threshold and the voltage slope value is greater than the slope applicable threshold, the static converter is selected as the voltage regulation device; otherwise, the mechanical tap transformer is selected as the voltage regulation device;

[0020] The selected voltage regulator type and voltage deviation direction are combined into an adjustment drive parameter group, which is sent to the execution interface of the corresponding voltage regulator to output an adjusted feedback voltage data sequence.

[0021] In a preferred embodiment, the feedback optimization module is used to calculate the voltage stabilization time, voltage residual amplitude and voltage oscillation amplitude through the adjusted feedback voltage data sequence to form an adjustment result evaluation index group;

[0022] Calculate the difference between the adjustment result evaluation index group and the cached adjustment response score value to obtain the strategy execution error value, which is used as the input for updating the strategy structure;

[0023] In the strategy structure update, the weight parameters in the adjustment strategy network are updated according to the strategy execution error value, thereby completing the structural correction of the feedback control strategy of the voltage regulation device.

[0024] In a preferred embodiment, a voltage disturbance feature vector extraction formula is constructed in the feature extraction module:

[0025] ;

[0026] Construct the fuzzy deviation paragraph judgment formula:

[0027] ;

[0028] The voltage time series Indicates the output end of the power equipment at the time point The actual voltage value at It is from Select the The first paragraph in the window The voltage value of each point; Indicates the The starting time point of each window segment; Indicates the length of the time window; Indicates the The median local voltage within a window segment; represents the disturbance judgment threshold; Indicates the The number of data points that meet the perturbation condition in a time window; Indicates the interval length used to calculate the voltage slope value; Indicates the The upper limit of the corrected slope value within a time window; Indicates the preset disturbance duration threshold; Indicates the preset voltage slope threshold; Indicates the The voltage disturbance characteristic vector corresponding to the time window; Indicates the binary judgment result of the fuzzy deviation paragraph, 1 indicates a fuzzy deviation paragraph. 0 indicates a non-ambiguous paragraph; Represents a Boolean indicator function, which takes a value of 1 when the condition in the brackets of the Boolean indicator function is met, otherwise it takes a value of 0; is the displacement sensitivity weighting factor of the voltage slope value.

[0029] In a preferred embodiment, in the strategy decision module, a strategy training objective function is constructed:

[0030] ;

[0031] Constructing the reward function:

[0032] ;

[0033] Generate control signals:

[0034] ;

[0035] in The disturbance number is The voltage disturbance eigenvector of ; Adjust the policy network to the target obtained after training; is any adjustment strategy function; Represents the target adjustment strategy network pair The output adjustment response score value; Indicates that in all In , we search for the adjustment strategy function that makes the expected return reach the upper limit; is the future return discount factor; in Indicates the direction from the current decision point to the future strategy step length; The cumulative window step for the return; is the strategy reward function; Number the disturbance The adjusted state vector, include and ; The time it takes for the regulated voltage to reach the target steady state; is the voltage deviation amplitude in the target stable state after adjustment; The reward fusion coefficients used to control the influence weights of different reward items; is the reward activation function; It is a very small positive number, which is used to prevent the denominator from being zero; The disturbance number is The adjustment response score value of The activation threshold for the preset regulation; It is the flag for regulating execution.

[0036] In a preferred embodiment, in the path selection module, a duration applicable threshold expression is constructed:

[0037] ;

[0038] Construct the slope applicable threshold expression:

[0039] ;

[0040] Construct a function to determine the type of regulating device:

[0041] ;

[0042] To build a parameter group for regulating the drive:

[0043] ;

[0044] in Number the disturbances; Number the historical samples; is the total number of historical samples; The disturbance number is Duration of disturbance; The disturbance number is The voltage slope value; Indicates the duration applicable threshold calculated under the current policy; Indicates the slope applicable threshold calculated under the current strategy; Sample number Voltage stabilization time after adjustment; Sample number A Boolean flag indicating whether it is a valid adjustment sample; Sample number The upper limit of the corrected slope value; Sample number The voltage fluctuation amplitude; Indicates the selection flag of the voltage regulator type. 1 indicates a static converter, 0 indicates a mechanical tap transformer; represents the sensitivity adjustment factor used to construct the applicable threshold for duration; represents the sensitivity adjustment factor used to construct the applicable threshold of the slope; Indicates the output voltage value of the power equipment in the corresponding section with disturbance number k; Set voltage targets for electrical equipment; is a sign function, which indicates the direction of voltage deviation; To adjust the drive parameter group, adjust the drive parameter group by Composed of voltage deviation direction.

[0045] Technical effects and advantages of the present invention:

[0046] 1. By introducing a feature recognition mechanism consisting of disturbance duration and voltage slope into the control system, fuzzy deviations can be identified and classified, solving the control error problem of the existing voltage regulation system that cannot effectively respond to atypical disturbances;

[0047] 2. By building a deep reinforcement learning regulation strategy network driven by stability indicators, we can achieve automatic generation of state-action mapping structures and continuous optimization of self-learning control strategies in voltage regulation systems.

[0048] 3. Dynamically select different types of voltage regulators based on disturbance characteristics and applicable thresholds, improving the adaptability of the regulation path to timeliness and stability in the automatic control system;

[0049] 4. Introducing a feedback evaluation index group consisting of voltage residual, oscillation amplitude and stabilization time to form a quantitative evaluation mechanism for the regulation quality in feedback control;

[0050] 5. An activation control mechanism based on the regulation response score was established to implement a closed-loop logic between the regulation process triggering and the linkage with the voltage regulator, improving the system's real-time adaptability to voltage fluctuation scenarios.

[0051] 6. The strategy structure update module is used to achieve adaptive adjustment of the weights of the regulation strategy network, thereby enhancing the evolutionary response capability of the control structure in the automatic control system to the long-term dynamic disturbance environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Schematic diagram of the system module of the present invention. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] Refer to the instruction manual Figure 1 , an embodiment of the present invention is a power equipment control system combined with deep reinforcement learning, including a feature extraction module, a strategy determination module, a path selection module, and a feedback optimization module;

[0055] The feature extraction module is used to perform structured disturbance analysis on the voltage time series data at the output of the power equipment. By extracting the disturbance duration and voltage slope value and comparing them with the preset threshold, it identifies the fuzzy deviation segments with feedback regulation significance.

[0056] The strategy decision module is used to construct a strategy mapping relationship between disturbance characteristics and regulation behaviors, and to determine whether to generate a control signal to drive feedback regulation based on the regulation response score.

[0057] The path selection module is used to determine whether to enter the regulation process based on the regulation execution flag, and select the corresponding voltage regulation device type based on the disturbance duration and voltage slope value in the voltage disturbance feature vector and the corresponding applicable threshold value. It also constructs a regulation drive parameter group based on the voltage deviation direction and sends it to the voltage regulation device to perform feedback regulation control;

[0058] The feedback optimization module is used to construct a regulation result evaluation index group based on the regulated feedback voltage data, and generate a strategy execution error value by calculating the difference with the regulation response score value, which is used to update the regulation strategy network to complete the feedback correction of the voltage regulation device.

[0059] The feature extraction module is used to perform real-time acquisition operations on the voltage time series data at the output end of the power equipment, obtain continuous time window segments, and then perform range calculation and local derivative operations on each segment in the continuous time window segment to output the voltage disturbance feature vector; the voltage disturbance feature vector includes the disturbance duration and voltage slope value;

[0060] The disturbance duration and voltage slope value are compared with the preset disturbance duration threshold and voltage slope threshold respectively; if the disturbance duration is less than the disturbance duration threshold and the voltage slope value is greater than the voltage slope threshold, the paragraph is marked as a fuzzy deviation paragraph, otherwise it is marked as a non-fuzzy paragraph;

[0061] During the fuzzy deviation paragraph labeling process, each set of voltage disturbance feature vectors and its corresponding manual judgment adjustment effect mark are bound as a sample pair, and the disturbance response dataset for strategy training is output. The manual judgment adjustment effect mark is based on the output voltage performance after adjustment. The operator determines whether the adjustment behavior is effective and assigns positive and negative labels as supervision signals for strategy learning.

[0062] The strategy decision module is used to input the disturbance response data set into the deep reinforcement learning network to construct a state-action mapping structure during the strategy training operation. It then updates the network weights based on the stability index composed of the regulated voltage stabilization time and the residual deviation amplitude as a reward function, and outputs the regulation strategy network.

[0063] In the execution strategy reasoning operation, the newly input voltage disturbance feature vector is input into the regulation strategy network to obtain the corresponding regulation response score value. If the regulation response score value is greater than the preset regulation activation threshold, the output regulation execution flag is activated, otherwise it is inactivated.

[0064] In executing the regulation execution flag output operation, the output regulation execution flag is sent as a control signal, and the current voltage disturbance feature vector and the regulation response score value are cached together for subsequent calls.

[0065] The path selection module is used to determine whether the regulation execution flag is in an active state. If it is in an active state, the regulation process is entered. If it is in an inactive state, the control state of the existing voltage regulation device in the power equipment is maintained unchanged and the operation of sending the regulation control signal is skipped;

[0066] According to the disturbance duration and voltage slope value in the voltage disturbance characteristic vector, the disturbance duration and voltage slope value are compared with the preset duration applicable threshold and slope applicable threshold respectively. If the disturbance duration is less than the duration applicable threshold and the voltage slope value is greater than the slope applicable threshold, the static converter is selected as the voltage regulation device; otherwise, the mechanical tap transformer is selected as the voltage regulation device;

[0067] The selected voltage regulator type and voltage deviation direction are combined into a regulation drive parameter group, which is sent to the corresponding voltage regulator's execution interface to output a regulated feedback voltage data sequence. Voltage regulator types include mechanical tap transformers and static converters. The voltage deviation direction refers to the direction in which the current output voltage deviates from the preset target voltage, i.e., upward (upward deviation) or downward (downward deviation).

[0068] It should be noted that the preset duration applicable threshold and slope applicable threshold are thresholds used to select the applicable type of physical regulating device. Based on the evaluation of the regulating effect of various regulating devices under different disturbance structures, the threshold range applicable to the device selection judgment can be determined through performance boundary identification and control stability requirements; however, the preset disturbance duration threshold and voltage slope threshold are thresholds used to classify whether the disturbance segment is a fuzzy deviation. By performing cluster analysis and label correspondence analysis on historical voltage disturbance data, combined with the manually labeled regulation effectiveness judgment results, the characteristic threshold or range threshold that can best distinguish between fuzzy and non-fuzzy disturbance segments can be manually selected; in actual applications, although both use disturbance duration and voltage slope values as judgment variables, they correspond to different control objectives respectively. Therefore, the judgment thresholds adopted are parameter groups independently set according to their respective application purposes, and their numerical settings and judgment meanings are not common.

[0069] The feedback optimization module is used to calculate the voltage stabilization time, voltage residual amplitude, and voltage oscillation amplitude from the adjusted feedback voltage data sequence to form a set of adjustment result evaluation indicators. In practical applications, continuous sample segments for evaluating the adjustment effect are first extracted from the feedback voltage time series after the adjustment behavior is executed. The shortest continuous time it takes for the output voltage to enter a stable state is determined based on the set voltage stability threshold, which is used as the voltage stabilization time. The voltage residual amplitude is then obtained by calculating the absolute difference between the voltage value in this stable state and the target voltage value. Finally, the maximum fluctuation amplitude of the difference change rate between all adjacent sampling points in the adjustment window is calculated as the voltage oscillation amplitude. The combination of these three indicators constitutes the adjustment result evaluation indicator group.

[0070] The difference between the regulation result evaluation index group and the cached regulation response score value is calculated to obtain the strategy execution error value, which is used as the input for updating the strategy structure. In actual applications, based on the correspondence between the voltage stability performance (representing the voltage stabilization time, voltage residual amplitude, and voltage oscillation amplitude) in the regulation result evaluation index group and the regulation response score value, a loss function is constructed to map the difference between the score value and the expected result. For example, when the regulation response score value is high but the corresponding voltage stability performance still has a large residual or a long stabilization time, it is considered that there is a deviation between the strategy output and the actual regulation effect. Based on this, the difference between the response score and the actual evaluation result is calculated and output as the strategy structure error value.

[0071] In the strategy structure update, the weight parameters in the regulation strategy network are updated according to the strategy execution error value to complete the structural correction of the feedback control strategy of the voltage regulation device; in practical applications, the strategy execution error value is input into the trained regulation strategy network as a loss signal, and the parameter gradient update process is executed through the back propagation algorithm to adjust the weight structure associated with the specific disturbance feature vector in the network, thereby correcting the response output of the regulation strategy under similar input conditions, making it closer to the voltage stability target in future execution, and completing the structural self-optimization of the feedback control strategy.

[0072] It should be noted that in the formula structure involved in this solution, dimensionless terms can serve as proportionality or structural adjustment factors. When combined with quantities with units, they only play a numerical scaling role and do not introduce new physical dimensions. Therefore, they will not change or confuse the overall unit system of expression. This combination of "dimensionless terms and units" can be understood as a composite structural expression commonly used in mathematical and physical modeling, conforming to the principle of dimensional consistency and having a clear physical interpretation basis.

[0073] Secondly, in the formula structure of this scheme, if multiple variables with different physical units are involved, including but not limited to time, mass or energy variables, their joint appearance is to express the collaborative modeling relationship of multiple physical mechanisms. Each variable can be formed into a unified structure through function mapping, ratio combination or normalization adjustment. The units and meanings are clear, and the overall expression conforms to the principle of dimensional consistency and the common formula of engineering modeling.

[0074] In this solution, any design constants, weights, adjustment factors, threshold parameters, and proportional coefficients are adjustable control parameters for different application environments. Their values depend on the target device configuration, data input characteristics, and performance optimization goals. During the implementation phase, they are set within a reasonable range through model verification, performance constraints, or engineering calibration. Although these parameters do not have unique preset values, they have clear adjustment logic and calculation paths, and are part of the deterministic setting process in engineering implementation. The purpose of such setting is to ensure that the solution is both universally adaptable, reproducible, and operable, without affecting its technical clarity and feasibility.

[0075] Construct the voltage disturbance feature vector extraction formula in the feature extraction module:

[0076] ;

[0077] Construct the fuzzy deviation paragraph judgment formula:

[0078] ;

[0079] The voltage time series Indicates the output end of the power equipment at the time point The actual voltage value at , in volts; It is from Select the The first paragraph in the window The voltage value of each point; Indicates the The starting time point of each window segment (in sampling points); Indicates the length of the time window, which is the observation range of each section of voltage disturbance data (in the number of sampling points); Indicates the The local voltage median within a window segment is used to capture the central baseline of the static disturbance and is defined as the median of the voltage sequence within the window (in volts). The disturbance judgment threshold is used to identify the fluctuation mutation boundary of voltage data within the window segment (in volts). Indicates the The number of data points that meet the perturbation condition in the time window, The number of data points that meet the disturbance condition in a time window is used to reflect the duration of the disturbance (in points); Indicates the interval length (in sampling points) used to calculate the voltage slope value, such as the differential distance between two points; Indicates the The upper limit of the correction slope value within a time window is used to characterize the voltage slope value (in volts / point); the voltage slope value is calculated by introducing a displacement sensitivity weighting factor of the voltage slope value. Enhance the sensitivity of the mutation starting position, thus different from the traditional differential; point Indicates the local index position within the current time window, used to Traverse the sampling points point by point; Indicates the preset disturbance duration threshold (in points); Indicates the preset voltage slope threshold (in volts / point); Indicates the The voltage disturbance characteristic vector corresponding to the time window includes and ; Indicates the binary judgment result of the fuzzy deviation paragraph, 1 indicates a fuzzy deviation paragraph. 0 indicates a non-ambiguous paragraph; Represents a Boolean indicator function, which takes a value of 1 when the condition in the brackets of the Boolean indicator function is met, otherwise it takes a value of 0; is the displacement sensitivity weighting factor of the voltage slope value. The displacement sensitivity weighting factor of the voltage slope value introduces a coupling judgment mechanism of time position and disturbance gradient to improve the recognition rate of the slope for early mutations. In addition, middle, The left side of the plus one is used to ensure that When the expression is not , The addition of one to the right of is used to ensure that the position weight in the time window increases from 2 instead of 1, so as to keep the displacement sensitivity weighting factor of the voltage slope value more smoothly distributed within the window and slightly widen the discrimination between different positions, thereby improving the recognition and discrimination of the front-segment mutation and the middle-segment mutation, avoiding too strong position concentration effect during training, and allowing the model to maintain differentiability and numerical stability while enhancing the early position response.

[0080] In the strategy decision module, construct the strategy training objective function:

[0081] ;

[0082] Constructing the reward function:

[0083] ;

[0084] Generate control signals:

[0085] ;

[0086] in The disturbance number is The voltage disturbance eigenvector of and feature extraction module Output is consistent, using different subscripts and symbols to avoid semantic conflicts; is the target optimal regulation strategy network obtained after training, the input of the target optimal regulation strategy network is the disturbance response data set, and the output of the target optimal regulation strategy network is the regulation response score value; is any feasible adjustment policy function, and any feasible adjustment policy function is used for policy space search; Represents the target optimal adjustment strategy network pair The output adjustment response score value; Indicates that in all In , we search for the adjustment strategy function that makes the expected return reach the upper limit; is the future return discount factor (unitless, range ), the future return discount factor is used in the strategy training objective function to gradually reduce the impact of future returns on current decisions, thereby balancing the weights of immediate returns and long-term returns; in Indicates the direction from the current decision point to the future strategy step length; Indicates that this step will be reported according to of The power is decayed to control the contribution ratio of the future return to the overall return; is the reward accumulation window step (unit: number of steps), which represents the forward strategy optimization length; is the strategy reward function. The larger the value of the strategy reward function, the better the current adjustment behavior. Number the disturbance The adjusted state vector, include and ; The time it takes for the regulated voltage to reach the target stable state (unit: sampling point); is the voltage deviation amplitude of the target stable state after adjustment (unit: volt); The reward fusion coefficients used to control the influence weights of different reward items; The reward activation function is used to scale and smooth the reward value. In practical applications, the reward activation function includes but is not limited to ; It is a very small positive number. It is used to prevent the denominator from being zero and causing numerical instability. In practical applications, The values include ; The disturbance number is The adjustment response score value is the continuous output of the policy network (real number); The preset adjustment activation threshold (the unit is consistent with the adjustment response score value), and the adjustment is executed when this value is exceeded; The adjustment execution flag is a binary Boolean value that indicates whether the adjustment control signal is actually issued.

[0087] In the path selection module, the build duration is subject to the following threshold expression:

[0088] ;

[0089] Construct the slope applicable threshold expression:

[0090] ;

[0091] Construct a function to determine the type of regulating device:

[0092] ;

[0093] To build a parameter group for regulating the drive:

[0094] ;

[0095] in Number the disturbances; Number the historical samples; is the total number of historical samples; The disturbance number is The duration of the disturbance (unit: sampling point), From the strategy decision module ; The disturbance number is The voltage slope value (unit: volts / point) comes from the strategy decision module ; Indicates the duration applicable threshold calculated under the current policy (unit: sampling point); Indicates the slope applicable threshold calculated under the current strategy (unit: volts / point); Sample number The voltage stabilization time after adjustment (unit: sampling point), The duration calculated from the feedback voltage data sequence analyzed and adjusted in the feedback optimization module is used to describe the time it takes for the voltage to recover from a disturbance state to a stable state; Sample number A Boolean flag indicating whether the sample is a valid adjustment (1 means the adjustment is valid, otherwise it is invalid); Sample number The upper limit of the corrected slope value (unit: volts / point) is used to evaluate the response strength; Sample number The voltage fluctuation amplitude (unit: volt) is used to correct the slope value suppression item; A selection flag indicating the type of voltage regulator (Boolean), 1 indicates a static converter, 0 indicates a mechanical tap transformer; It represents the sensitivity adjustment factor used to construct the applicable threshold of the duration, and is used to control the influence weight of the disturbance duration in the device selection; Indicates the sensitivity adjustment factor used to construct the slope applicable threshold, which is used to adjust the threshold scale of the voltage slope value in the strategy judgment; in actual application and The value of can be empirically adjusted based on the stability of the device switching effect in historical adjustment samples, including but not limited to selecting a value within the range of 0.5 to 2 to achieve balanced adjustment of the threshold response; Indicates the output voltage value of the power equipment in the corresponding section of the disturbance number k (unit: V); Set the voltage target for the power device (unit: volts); The sign function indicates the direction of voltage deviation: a sign value of +1 indicates that the voltage is higher than the target voltage (upper deviation), and a sign value of -1 indicates that the voltage is lower than the target voltage (lower deviation). To adjust the drive parameter group, adjust the drive parameter group by It is composed of the voltage deviation direction and sent to the execution interface of the voltage regulation device as a control signal.

[0096] It should be noted that this solution focuses on the disturbance response regulation problem in power equipment. It constructs a four-stage control structure consisting of a "feature extraction module," a "strategy determination module," a "path selection module," and a "feedback optimization module," forming a regulation control process with adaptive strategy evolution capabilities.

[0097] During actual power equipment operation, the voltage output terminal may experience disturbance waveforms due to load fluctuations, external interference, or nonlinear system responses. These disturbances may not reach traditional fault thresholds, but they may lead to a decrease in system stability. Therefore, the first stage, the "feature extraction module," collects voltage time series data in real time, divides it into continuous time window segments, and performs range calculations (to capture the voltage fluctuation amplitude) and local derivative operations (to reflect voltage change trends) on each segment. This constructs an expressive "voltage disturbance feature vector," which includes two key indicators: disturbance duration and voltage slope value. These disturbance features are then compared with preset thresholds to mark fuzzy deviation segments. Combined with manual adjustment effect annotations, a "disturbance response dataset" is formed to provide label supervision support for subsequent learning.

[0098] In the second stage, the policy decision module uses the disturbance response dataset as input to construct a state-action mapping structure in reinforcement learning, with the voltage disturbance feature vector as the state input and the regulation behavior as the action output, forming a policy network. The training goal of this policy network is to maximize the voltage stability performance after regulation. Specifically, the stability index composed of "voltage stabilization time" and "residual deviation amplitude" is used as the reward function, so that the policy can prioritize learning a path that quickly responds to the initial disturbance and reduces the deviation. During the policy reasoning process, when a new disturbance feature vector is input, the system will adjust the response score value based on the existing network output to determine whether to activate the regulation behavior. If the score exceeds the activation threshold, the regulation execution flag is issued and the current disturbance and response results are cached for subsequent optimization.

[0099] In the third stage, after determining that regulation is necessary, the path selection module does not directly execute a single path. Instead, it performs a dual judgment based on the disturbance duration and voltage slope values in the current disturbance characteristics, combined with the "duration threshold" and "slope threshold." This threshold group is set to reflect the adaptability of different physical devices to specific disturbance forms. For example, short disturbances with steep voltage slopes are suitable for fast-responding static converters, while slowly changing and gently sloping disturbances are suitable for more stable but slower-responding mechanical tap transformers. After the selected regulation device type is finally determined in this stage, the "voltage deviation direction" (i.e., whether the current voltage is higher or lower than the target voltage) is combined to form a regulation drive parameter group, which is sent to the selected device interface to execute the specific regulation operation.

[0100] The fourth stage, the "Feedback Optimization Module," uses the post-regulation voltage sequence data to calculate three indicators: "voltage stabilization time," "voltage residual amplitude," and "voltage oscillation amplitude," forming an evaluation index group. This group of indicators comprehensively reflects the dynamic characteristics of the voltage after regulation. This evaluation result is then compared with the previously cached regulation response score to calculate the error in the strategy execution value. This error serves as feedback on the current strategy's performance in the real system. The system then uses this error in the feedback update process of the regulation strategy network, automatically correcting the weight parameters in the strategy structure, thus completing a closed-loop evolution of the strategy structure and achieving self-reinforcement and iterative convergence of the regulation control strategy.

[0101] Traditional voltage regulation methods often rely on rule thresholds and preset logic, and are unable to cope with complex disturbance forms and nonlinear dynamic changes. However, this solution uses an end-to-end feature extraction and strategy learning mechanism to achieve real-time recognition and layered response while introducing an enhanced feedback path based on feedback errors. This achieves dynamic adaptation of regulation capabilities and multiple rounds of correction of network strategies. It is suitable for disturbance suppression and regulation optimization tasks in scenarios such as new power equipment and smart grid scheduling.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A power equipment control system combined with deep reinforcement learning, comprising a feature extraction module, a strategy determination module, a path selection module, and a feedback optimization module, characterized by: The feature extraction module is used to perform structured disturbance analysis on the voltage time series data at the output of the power equipment. By extracting the disturbance duration and voltage slope value and comparing them with the preset threshold, it identifies the fuzzy deviation segments with feedback regulation significance. The strategy decision module is used to construct a strategy mapping relationship between disturbance characteristics and regulation behaviors, and to determine whether to generate a control signal to drive feedback regulation based on the regulation response score. The path selection module is used to determine whether to enter the regulation process based on the regulation execution flag, and select the corresponding voltage regulation device type based on the disturbance duration and voltage slope value in the voltage disturbance feature vector and the corresponding applicable threshold value. It also constructs a regulation drive parameter group based on the voltage deviation direction and sends it to the voltage regulation device to perform feedback regulation control; The feedback optimization module is used to construct a regulation result evaluation index group based on the regulated feedback voltage data, and calculate the difference between the regulated response score and the strategy execution error value, which is used to update the regulation strategy network to complete the feedback correction of the voltage regulation device; The strategy decision module is used to input the disturbance response data set into the deep reinforcement learning network to construct a state-action mapping structure during the strategy training operation. It then updates the network weights based on the stability index composed of the regulated voltage stabilization time and the residual deviation amplitude as a reward function, and outputs the regulation strategy network. In the execution strategy reasoning operation, the newly input voltage disturbance feature vector is input into the regulation strategy network to obtain the corresponding regulation response score value. If the regulation response score value is greater than the preset regulation activation threshold, the output regulation execution flag is activated, otherwise it is inactivated. In the operation of outputting the adjustment execution flag, the output adjustment execution flag is sent as a control signal, and the current voltage disturbance feature vector and the adjustment response score value are cached together for subsequent calls; The feedback optimization module is used to calculate the voltage stabilization time, voltage residual amplitude and voltage oscillation amplitude through the adjusted feedback voltage data sequence to form an adjustment result evaluation index group; Calculate the difference between the adjustment result evaluation index group and the cached adjustment response score value to obtain the strategy execution error value, which is used as the input for updating the strategy structure; In the strategy structure update, the weight parameters in the adjustment strategy network are updated according to the strategy execution error value, thereby completing the structural correction of the feedback control strategy of the voltage regulation device.

2. The power equipment control system combined with deep reinforcement learning according to claim 1, characterized in that: The feature extraction module is used to perform real-time acquisition operations on the voltage time series data at the output end of the power equipment, obtain continuous time window segments, and then perform range calculation and local derivative operations on each segment in the continuous time window segment to output the voltage disturbance feature vector; the voltage disturbance feature vector includes the disturbance duration and voltage slope value; The disturbance duration and voltage slope value are compared with the preset disturbance duration threshold and voltage slope threshold respectively; if the disturbance duration is less than the disturbance duration threshold and the voltage slope value is greater than the voltage slope threshold, the paragraph is marked as a fuzzy deviation paragraph, otherwise it is marked as a non-fuzzy paragraph; During the fuzzy deviation paragraph labeling process, each set of voltage disturbance feature vectors and their corresponding manual judgment adjustment effect labels are bound as sample pairs, and a disturbance response dataset for strategy training is output.

3. The power equipment control system combined with deep reinforcement learning according to claim 2, characterized in that: The path selection module is used to determine whether the regulation execution flag is in an active state. If it is in an active state, the regulation process is entered. If it is in an inactive state, the control state of the existing voltage regulation device in the power equipment is maintained unchanged and the operation of sending the regulation control signal is skipped; According to the disturbance duration and voltage slope value in the voltage disturbance characteristic vector, the disturbance duration and voltage slope value are compared with the preset duration applicable threshold and slope applicable threshold respectively. If the disturbance duration is less than the duration applicable threshold and the voltage slope value is greater than the slope applicable threshold, the static converter is selected as the voltage regulation device; otherwise, the mechanical tap transformer is selected as the voltage regulation device; The selected voltage regulator type and voltage deviation direction are combined into an adjustment drive parameter group, which is sent to the execution interface of the corresponding voltage regulator to output an adjusted feedback voltage data sequence.

4. The power equipment control system combined with deep reinforcement learning according to claim 3, characterized in that: Construct the voltage disturbance feature vector extraction formula in the feature extraction module: ； Construct the fuzzy deviation paragraph judgment formula: ； The voltage time series Indicates the output end of the power equipment at the time point The actual voltage value at It is from Select the The first paragraph in the window The voltage value of each point; Indicates the The starting time point of each window segment; Indicates the length of the time window; Indicates the The median local voltage within a window segment; represents the disturbance judgment threshold; Indicates the The number of data points that meet the perturbation condition in a time window; Indicates the interval length used to calculate the voltage slope value; Indicates the The upper limit of the corrected slope value within a time window; Indicates the preset disturbance duration threshold; Indicates the preset voltage slope threshold; Indicates the The voltage disturbance characteristic vector corresponding to the time window; Indicates the binary judgment result of the fuzzy deviation paragraph, 1 indicates a fuzzy deviation paragraph. 0 indicates a non-ambiguous paragraph; Represents a Boolean indicator function, which takes a value of 1 when the condition in the brackets of the Boolean indicator function is met, otherwise it takes a value of 0; is the displacement sensitivity weighting factor of the voltage slope value.

5. The power equipment control system combined with deep reinforcement learning according to claim 4, characterized in that: In the strategy decision module, construct the strategy training objective function: ； Constructing the reward function: ； Generate control signals: ； in The disturbance number is The voltage disturbance eigenvector of ; Adjust the policy network to the target obtained after training; is any adjustment strategy function; Represents the target adjustment strategy network pair The output adjustment response score value; Indicates that in all In , we search for the adjustment strategy function that makes the expected return reach the upper limit; is the future return discount factor; in Indicates the direction from the current decision point to the future strategy step length; The cumulative window step for the return; is the strategy reward function; Number the disturbance The adjusted state vector, include and ; The time it takes for the regulated voltage to reach the target steady state; is the voltage deviation amplitude in the target stable state after adjustment; The reward fusion coefficients used to control the influence weights of different reward items; is the reward activation function; It is a very small positive number, which is used to prevent the denominator from being zero; The disturbance number is The adjustment response score value of The activation threshold for the preset regulation; It is the flag for regulating execution.

6. The power equipment control system combined with deep reinforcement learning according to claim 5, characterized in that: In the path selection module, the build duration is subject to the following threshold expression: ； Construct the slope applicable threshold expression: ； Construct a function to determine the type of regulating device: ； To build a parameter group for regulating the drive: ； in Number the disturbances; Number the historical samples; is the total number of historical samples; The disturbance number is Duration of disturbance; The disturbance number is The voltage slope value; Indicates the duration applicable threshold calculated under the current policy; Indicates the slope applicable threshold calculated under the current strategy; Sample number Voltage stabilization time after adjustment; Sample number A Boolean flag indicating whether it is a valid adjustment sample; Sample number The upper limit of the corrected slope value; Sample number The voltage fluctuation amplitude; Indicates the selection flag of the voltage regulator type. 1 indicates a static converter, 0 indicates a mechanical tap transformer; represents the sensitivity adjustment factor used to construct the applicable threshold for duration; represents the sensitivity adjustment factor used to construct the applicable threshold of the slope; Indicates the output voltage value of the power equipment in the corresponding section with disturbance number k; Set voltage targets for electrical equipment; is a sign function, which indicates the direction of voltage deviation; To adjust the drive parameter group, adjust the drive parameter group by Composed of voltage deviation direction.

Images