Dual-time scale DRL cooperative voltage regulation control method and system

Abstract

The invention provides a dual-time-scale DRL cooperative voltage regulation control method and system, and relates to the technical field of power system voltage regulation control. According to the technical scheme provided by the invention, the two sub-models work cooperatively, the fast-scale sub-model provides fast response to guarantee the voltage regulation speed, and the slow-scale sub-model provides accurate regulation to guarantee the voltage regulation precision. A voltage regulation event is stored in a shared experience playback pool and a training sample is randomly extracted to update a collaborative decision model, so that the model continuously optimizes a decision strategy in deep reinforcement learning training, a control scheme with the optimal comprehensive performance is finally obtained through convergence, the voltage regulation speed and precision can be considered, and the voltage regulation effect is improved.

Description

Technical Field

[0001] This application relates to the field of power system voltage regulation control technology, specifically to a dual-time-scale DRL collaborative voltage regulation control method and system. Background Technology

[0002] Voltage regulation refers to the technical means of controlling the voltage at power grid nodes using voltage regulating equipment to maintain it within a specified range. During power system operation, factors such as load fluctuations and changes in distributed power generation output cause the voltage at power grid nodes to deviate from its rated value. Excessive voltage damages electrical equipment, while insufficient voltage affects the normal operation of equipment; therefore, voltage regulating equipment is needed to adjust the power grid voltage in real time.

[0003] Most voltage regulation methods in related technologies involve independent control of a single device, meaning that a specific voltage regulator is selected to perform regulation operations for voltage fluctuations. This approach suffers from a limitation in practical applications: fast voltage regulators respond quickly but have limited regulation precision, resulting in unsatisfactory performance in scenarios requiring precise control. Slow voltage regulators offer high regulation precision but suffer from a delay, during which voltage fluctuations cannot be controlled in a timely manner. Independent control of a single device cannot simultaneously meet the requirements of both response speed and regulation precision. In complex and variable voltage fluctuation scenarios (such as distribution networks containing distributed photovoltaic, wind power, or mobile energy storage), the voltage regulation process either results in a slow response leading to prolonged voltage exceedances or coarse regulation resulting in insufficient voltage control precision, failing to balance speed and accuracy, and thus exhibiting poor voltage regulation performance. Summary of the Invention

[0004] This application provides a dual-timescale DRL collaborative voltage regulation control method and system, which can balance voltage regulation speed and accuracy, and improve the voltage regulation effect.

[0005] Firstly, this application provides a dual-timescale DRL coordinated voltage regulation control method, the method comprising: A collaborative decision-making model for dual-timescale DRL is constructed, which includes a fast-scale sub-model and a slow-scale sub-model that share an experience replay pool. Real-time collection of voltage data from power grid nodes, and input of the voltage data into the collaborative decision-making model; The fast-scale sub-model in the collaborative decision-making model is used to monitor voltage data routinely, and a fast voltage regulation command is output when the voltage data exceeds the preset voltage safety range. The fast voltage regulation command is used to control the fast voltage regulation equipment to regulate the grid voltage. When the voltage data is detected to meet the preset triggering conditions, the slow-scale sub-model in the collaborative decision-making model is activated to output a slow voltage regulation command until the adjusted target voltage data is restored to the voltage safety range. The slow voltage regulation command is used to control the slow voltage regulation equipment to regulate the grid voltage. Voltage regulation events are stored as training samples in a shared experience replay pool, and training samples are randomly drawn from the shared experience replay pool to update the collaborative decision-making model until the collaborative decision-making model converges.

[0006] By adopting the above technical solution, a collaborative decision-making model including a fast-scale sub-model and a slow-scale sub-model is constructed, realizing the collaborative control of voltage regulation devices with different response characteristics. The fast-scale sub-model monitors voltage data routinely, and when the voltage exceeds the safe range, it immediately outputs a fast voltage regulation command to control the fast voltage regulation device, quickly responding to voltage fluctuations and achieving timely adjustment. When the voltage data meets preset trigger conditions, the slow-scale sub-model is activated to output a slow voltage regulation command to control the slow voltage regulation device, utilizing the large-capacity adjustment capability of the slow voltage regulation device to achieve precise control until the voltage returns to the safe range. The two sub-models work together, with the fast-scale sub-model providing fast response to ensure voltage regulation speed, and the slow-scale model providing precise adjustment to ensure voltage regulation accuracy. By storing voltage regulation events in a shared experience replay pool and randomly sampling training samples to update the collaborative decision-making model, the model continuously optimizes the decision-making strategy during deep reinforcement learning training, ultimately converging to obtain a control scheme with optimal comprehensive performance, which can balance voltage regulation speed and accuracy, improving the voltage regulation effect.

[0007] Secondly, this application provides a dual-time-scale DRL coordinated voltage regulation control system, the system comprising: The model building module is used to build a collaborative decision-making model for dual-timescale DRL. The collaborative decision-making model includes a fast-scale sub-model and a slow-scale sub-model that share an experience replay pool. The voltage acquisition module is used to collect voltage data from power grid nodes in real time and input the voltage data into the collaborative decision-making model; The fast voltage regulation module is used to monitor voltage data routinely using the fast-scale sub-model in the collaborative decision-making model, and outputs a fast voltage regulation command when the voltage data exceeds the preset voltage safety range. The fast voltage regulation command is used to control the fast voltage regulation equipment to regulate the grid voltage. The slow voltage regulation module is used to activate the slow-scale sub-model in the collaborative decision-making model to output a slow voltage regulation command when the monitored voltage data meets the preset trigger conditions, until the adjusted target voltage data is restored to the voltage safety range. The slow voltage regulation command is used to control the slow voltage regulation equipment to regulate the grid voltage. The model update module is used to store voltage regulation events as training samples in the shared experience replay pool, and randomly select training samples from the shared experience replay pool to update the collaborative decision-making model until the collaborative decision-making model converges.

[0008] Thirdly, this application provides a computer storage medium that stores multiple instructions adapted for loading by a processor and executing any of the methods described above.

[0009] Fourthly, this application provides an electronic device including a processor, a memory, and a transceiver. The memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform any of the methods described above.

[0010] In summary, the beneficial effects of the technical solution of this application include: By adopting the above technical solution, a collaborative decision-making model including a fast-scale sub-model and a slow-scale sub-model is constructed, realizing the collaborative control of voltage regulation devices with different response characteristics. The fast-scale sub-model monitors voltage data routinely, and when the voltage exceeds the safe range, it immediately outputs a fast voltage regulation command to control the fast voltage regulation device, quickly responding to voltage fluctuations and achieving timely adjustment. When the voltage data meets preset trigger conditions, the slow-scale sub-model is activated to output a slow voltage regulation command to control the slow voltage regulation device, utilizing the large-capacity adjustment capability of the slow voltage regulation device to achieve precise control until the voltage returns to the safe range. The two sub-models work together, with the fast-scale sub-model providing fast response to ensure voltage regulation speed, and the slow-scale model providing precise adjustment to ensure voltage regulation accuracy. By storing voltage regulation events in a shared experience replay pool and randomly sampling training samples to update the collaborative decision-making model, the model continuously optimizes the decision-making strategy during deep reinforcement learning training, ultimately converging to obtain a control scheme with optimal comprehensive performance, which can balance voltage regulation speed and accuracy, improving the voltage regulation effect. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a dual-time-scale DRL coordinated voltage regulation control method according to an embodiment of this application. Figure 2 This is a schematic diagram of the structure of a dual-time-scale DRL collaborative voltage regulation control system according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0012] Explanation of reference numerals in the attached drawings: 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. Detailed Implementation

[0013] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0014] In the description of the embodiments of this application, words such as "illustrative," "for example," or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "illustrative," "for example," or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of words such as "illustrative," "for example," or "for example" is intended to present the relevant concepts in a specific manner.

[0015] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0016] Please see Figure 1 This is a flowchart illustrating a dual-time-scale DRL collaborative voltage regulation control method provided in an embodiment of this application. This method can be implemented using a computer program, a microcontroller, or run on a dual-time-scale DRL collaborative voltage regulation control system based on the von Neumann architecture. The computer program can be integrated into the application or run as a standalone utility application. The specific steps of the dual-time-scale DRL collaborative voltage regulation control method are described in detail below.

[0017] S101: Construct a collaborative decision-making model for dual-timescale DRL, which includes a fast-scale sub-model and a slow-scale sub-model that share an experience replay pool. The dual-timescale DRL collaborative decision-making model refers to an intelligent decision-making system built on deep reinforcement learning techniques that can operate collaboratively at different time scales. The fast-scale sub-model represents the decision network responsible for handling high-frequency voltage fluctuations. The slow-scale sub-model represents the decision network responsible for correcting steady-state voltage deviations. The shared experience replay pool is a cache area that centrally stores historical voltage regulation experience data; both sub-models sample data from this replay pool to update their parameters during training.

[0018] Specifically, this step is performed during the initialization phase of the power system voltage regulation control system. Its main purpose is to establish an intelligent decision-making framework capable of handling voltage disturbances at different time scales. First, the criteria for dividing the fast and slow time scales need to be determined based on the operating state parameters of the grid nodes and the equipment parameters of the voltage regulating equipment. For the fast-scale sub-model, its network structure needs to be designed to receive high-dimensional vectors containing time-series voltage data. After extracting time-series features through neural network layers, the control input for the fast voltage regulating equipment is output. For the slow-scale sub-model, its network structure needs to be designed to receive steady-state feature vectors. After processing through neural network layers, the control commands for the slow voltage regulating equipment are output. Both sub-models share an experience playback pool, which employs a specific data management strategy for sample storage and updating.

[0019] To further clarify the specific implementation of this step, the following examples illustrate the process: For instance, the timescale of the fast-scale sub-model can be set to 10 to 100 milliseconds, and a convolutional neural network (CNN) structure can be used to extract feature patterns from time-series voltage data. Similarly, the timescale of the slow-scale sub-model can be set to 2 to 3 seconds, and a multilayer perceptron (MLP) structure can be used to process steady-state voltage statistical features. The shared experience replay pool can be stored in a four-tuple format (current state, executed action, reward value, next state), and both sub-models randomly sample data from this replay pool during training to update their parameters. For example, when the grid experiences a sudden voltage drop due to a change in photovoltaic output, the fast-scale sub-model can output a reactive power regulation command for the static var generator (SVG) within 20 milliseconds, while the slow-scale sub-model determines that the tap position of the on-load tap changer (OLTC) needs to be adjusted after continuous monitoring for 30 milliseconds.

[0020] S102: Collect voltage data of power grid nodes in real time and input the voltage data into the collaborative decision-making model; Voltage data refers to real-time measured values ​​that reflect the voltage operating status of power grid nodes.

[0021] Specifically, this step involves real-time acquisition of voltage data using voltage sensors or power quality monitoring devices deployed at various monitoring nodes in the power grid. The acquired voltage data undergoes preprocessing, including data cleaning, outlier removal, and normalization, to meet the input format requirements of the collaborative decision-making model. The preprocessed voltage data is then organized into a state vector according to a predetermined data structure and input into the collaborative decision-making model. The fast-scale sub-model receives high-frequency sampled voltage time-series data, while the slow-scale sub-model receives statistically processed voltage feature data. The data input process must ensure real-time performance to guarantee that the model can make decisions based on the latest power grid conditions.

[0022] To further clarify the specific implementation of this step, an example is provided below: For instance, voltage sensors can be deployed at key nodes in the distribution network, with a sampling frequency set to 1000 times per second, i.e., collecting voltage data once per millisecond. The collected raw voltage data can include parameters such as three-phase voltage amplitude and phase angle, and the data format can be [V_A, V_B, V_C, θ_A, θ_B, θ_C]. The collected voltage data can be normalized to the [0, 1] interval, and the calculation formula can be V_norm=(V-V_min) / (V_max-V_min), where V_min and V_max are the minimum and maximum voltage boundaries, respectively, and can be set to 0.9 pu and 1.1 pu. The input to the fast-scale sub-model can be voltage time-series data containing the most recent 100 sampling points, forming a 100×6 time-series matrix, with the input data updated every 10 milliseconds. For example, the input to the slow-scale sub-model can be the statistical features of voltage data over the past 2 seconds, including the mean, standard deviation, maximum, minimum, etc., forming a feature vector with a dimension of 24 (assuming 6 nodes are monitored, and 4 statistical features for each node), and the input data is updated every 2 seconds.

[0023] S103: Use the fast-scale sub-model in the collaborative decision-making model to monitor voltage data routinely, and output a fast voltage regulation command when the voltage data exceeds the preset voltage safety range. The fast voltage regulation command is used to control the fast voltage regulation equipment to regulate the grid voltage. Routine monitoring refers to the process by which the fast-scale sub-model continuously analyzes and evaluates voltage data in real time at a high frequency. The preset voltage safety range refers to the allowable normal fluctuation range of the grid voltage; exceeding this range indicates a voltage deviation requiring adjustment. The rapid voltage regulation command refers to the control signal output by the fast-scale sub-model used to control fast-response voltage regulation equipment. Fast-response voltage regulation equipment refers to equipment capable of rapidly changing its output to regulate voltage within a short period.

[0024] Specifically, in this step, the fast-scale sub-model receives voltage data at a high sampling frequency and performs real-time inference calculations on each set of input data. Internally, the model uses a CNN (Convolutional Neural Network) structure to extract the temporal features of voltage fluctuations and determine whether the current voltage state is within a safe range. When the voltage data is detected to exceed the preset safe range, the fast-scale sub-model immediately selects an appropriate control action from its action space based on the magnitude and direction of the current voltage deviation, generating a rapid voltage regulation command. This command is transmitted to the rapid voltage regulation device through the control system. Upon receiving the command, the device quickly executes the corresponding adjustment action, pulling the voltage back to the safe range within a short time. The entire monitoring and response process needs to be completed within milliseconds.

[0025] To further clarify the specific implementation of this step, the following examples illustrate the concept: For instance, the preset voltage safety range can be set to V_std ± 5%, where V_std is the rated voltage, such as 9.5kV-10.5kV for a 10kV distribution network. For instance, the fast-scale sub-model samples the grid node voltage V_real every 10ms, i.e., the sampling frequency is 100Hz. For instance, fast voltage regulation equipment includes SVG (Static Var Generator) or STATCOM, where the SVG device's response time is ≤5ms and its rated capacity is ≥10Mvar. For instance, the fast-scale sub-model uses a CNN structure, with a 128-dimensional input layer (containing voltage values, voltage change rates, and SVG output capacity, trigger angle, and other device operating states for the past 10-20 time-series voltage points), two convolutional layers (3×3 kernel size), one fully connected layer (64 neurons), and the output layer outputting the SVG reactive power regulation amount ΔQ. For example, when the photovoltaic output fluctuation causes the grid node voltage V_real to drop instantaneously to 9.3kV (deviation -7%, exceeding the lower limit of the safe range of 9.5kV), the fast-scale sub-model outputs a fast voltage regulation command within 10ms after detecting the anomaly. The command content is the reactive power regulation amount ΔQ of the SVG. The SVG responds within 5ms after receiving the command and raises the voltage to 9.6kV (deviation -4%).

[0026] S104: When the voltage data is detected to meet the preset triggering conditions, the slow-scale sub-model in the collaborative decision-making model is activated to output a slow voltage regulation command until the adjusted target voltage data is restored to the voltage safety range. The slow voltage regulation command is used to control the slow voltage regulation equipment to regulate the grid voltage. The preset trigger condition refers to the specific voltage state condition used to determine whether slow-scale voltage regulation needs to be initiated. The slow-scale voltage regulation command refers to the control signal output by the slow-scale sub-model used to control the slow-response voltage regulation device. The slow-scale voltage regulation device refers to a device with a relatively long response time but high regulation accuracy, suitable for steady-state voltage correction. The target voltage data refers to the measured voltage value of the grid node after regulation by the voltage regulation device.

[0027] Specifically, in this step, the system continuously monitors whether the voltage data meets the preset trigger conditions. When the trigger conditions are detected, the coordination control module forcibly triggers the slow-scale sub-model to start. The slow-scale sub-model adopts an MLP (Multilayer Perceptron) structure, receiving the steady-state average voltage, grid load factor, and the current state of slow-speed equipment as inputs. Through neural network inference calculation, it outputs a slow-speed voltage regulation command. This command is sent to the slow-speed voltage regulation equipment for execution. Due to the inherent action delay of the slow-speed equipment, the fast-scale sub-model executes a pre-compensation strategy during the delay period. Based on the slow-scale command, it predicts the expected voltage change ΔV_pred and outputs a compensation command in advance to offset the voltage fluctuation during the delay period. After the slow-speed equipment takes effect, the system continues to monitor the adjusted target voltage data. If the target voltage recovers to a safe range, the system returns to the fast-scale dominant mode, and the slow-scale sub-model enters a dormant state.

[0028] To further clarify the specific implementation of this step, the following example illustrates the following: For instance, the preset trigger condition could be set as "the fast-scale sub-model fails to stabilize the voltage within a safe range for three consecutive sampling periods (30ms in total)" or "voltage deviation ≥ ±5% and continues to increase." For example, a slow-scale voltage regulation device includes an OLTC tap changer and fixed capacitor banks. The OLTC tap changer has 16 adjustment levels and an action delay of 2.5s. The high-voltage capacitor banks use a group switching method, with each group having a capacity of 2Mvar. For example, the slow-scale sub-model uses an MLP structure, with an 8-dimensional input layer (including the steady-state voltage mean, load factor, OLTC tap changer position, number of capacitor switching groups, etc.), two hidden layers (32 neurons per layer), and an output layer that outputs the OLTC tap changer adjustment level or the number of capacitor switching groups. For example, if the fast-scale sub-model increases the voltage from 9.3kV to 9.6kV for three consecutive sampling periods (30ms) but still fails to reach the lower limit of the safe range of 9.5kV, the slow-scale sub-model is triggered. The MLP sub-model outputs a command to increase the OLTC tap by one level (expected voltage increase ΔV_pred = 0.3kV). For example, during the 2.5s delay before the OLTC tap action takes effect, the fast-scale sub-model calculates the SVG pre-compensation amount ΔQ_comp = k × ΔV_pred (where k is the compensation coefficient, calibrated by the grid impedance characteristics) based on the expected voltage change ΔV_pred = 0.3kV, for example, ΔQ_comp = 2Mvar, to adjust in advance to raise the voltage to 9.8kV, offsetting the voltage fluctuation during the delay period. For example, after 2.5 seconds, the OLTC tap changer is adjusted and the target voltage V_real stabilizes at 10.0kV (0% deviation, returning to the safe range of 9.5kV-10.5kV). The system then returns to the fast-scale dominant mode, and the slow-scale sub-model enters a dormant state, monitoring the steady-state deviation only once every 2 seconds.

[0029] S105: Store voltage regulation events as training samples in the shared experience replay pool, and randomly select training samples from the shared experience replay pool to update the collaborative decision-making model until the collaborative decision-making model converges.

[0030] Voltage regulation events refer to single voltage regulation process records that occur during actual power grid operation, containing complete information such as the initial voltage state, the voltage regulation actions performed, and the resulting regulation effects. Training samples refer to structured data extracted from voltage regulation events for model training. Model convergence refers to the state where, after continuous online learning, the performance indicators of the collaborative decision-making model tend to stabilize and no longer improve significantly.

[0031] Specifically, after each voltage regulation event, the system automatically records the complete information of the event, organizes this information into a standard format for training samples, and stores it in a shared experience replay pool. Training samples generated by both the fast-scale and slow-scale sub-models are stored in the same shared experience replay pool, realizing an experience-sharing mechanism. During the online update phase of the model, the system periodically randomly selects a certain number of training samples from the shared experience replay pool, uses these samples to calculate the loss function, and updates the neural network parameters of the collaborative decision-making model through the backpropagation algorithm. By continuously accumulating new voltage regulation experience and continuously updating parameters, the model gradually optimizes its decision-making strategy. When the model's performance indicators remain stable over several consecutive update cycles, the model is considered to have converged.

[0032] To further clarify the specific implementation of this step, the following examples illustrate the process: For instance, the training samples are formatted as a (state-action-reward-next state) quadruple, where the state includes voltage data V_real, equipment status, etc., the action includes SVG reactive power regulation ΔQ, OLTC adjustment level, etc., and the reward value is calculated based on the reward function. For instance, the shared experience replay pool has a capacity of 100,000 samples, using a circular queue structure; when the pool is full, the oldest sample is automatically overwritten. For instance, the model is updated online every 5 minutes using 1000 newly collected real-time data sets for incremental updates to adapt to changes in the power grid's operating state. For instance, during model training, an ε-greedy strategy is used to balance exploration and utilization, alternating between training the CNN sub-model and the MLP sub-model, and randomly selecting training samples from the shared experience replay pool to update network parameters. For instance, the convergence criterion for the model is a voltage regulation error ≤ ±2%; that is, convergence is determined when the model's voltage regulation accuracy consistently reaches this standard. For example, in the voltage regulation event where the voltage was adjusted from 9.3kV to 10.0kV, the voltage regulation sample (including the initial state V_real=9.3kV, the fast voltage regulation action ΔQ, the slow voltage regulation action OLTC up by 1 level, the reward value, the final state V_real=10.0kV, etc.) was stored in the shared experience replay pool. The model then performed incremental updates, which reduced the voltage regulation error when encountering similar voltage fluctuations in the future from the initial -7% to ±1.5%, thus verifying the learning effect of the model.

[0033] Based on the above embodiments, as an optional implementation method, the method of time scale segmentation and voltage regulation device allocation before S101 can be implemented through the following steps S201-S203.

[0034] S201: Obtain the operating status parameters of the power grid nodes and the equipment parameters of multiple voltage regulating devices; Operating status parameters refer to the electrical characteristics of power grid nodes during actual operation, including data such as node voltage, current, and power factor. Equipment parameters refer to the technical specifications and performance indicators of voltage regulating equipment, including characteristic data such as equipment response time, rated capacity, and regulation range.

[0035] Specifically, this step acquires the operating status parameters of the power grid nodes through a voltage data acquisition module. The data acquisition module includes voltage and current sensors, with a sampling frequency exceeding 100Hz, collecting real-time data on voltage, current, and power factor of the power grid nodes. Simultaneously, it acquires the equipment parameters of multiple voltage regulating devices already connected to the system, including technical indicators such as response time, rated capacity, number of regulation levels, and action delay for each device. These operating status and equipment parameters provide fundamental data support for subsequent device segmentation and time scale settings, ensuring that the dual-time-scale voltage regulation control method can be accurately configured according to the actual state of the power grid and the performance characteristics of the equipment.

[0036] S202: Based on equipment parameters, multiple voltage regulating devices are divided into fast voltage regulating devices and slow voltage regulating devices; Specifically, based on the response time and action delay characteristics of each voltage regulator, multiple voltage regulators are divided into two categories: fast voltage regulators and slow voltage regulators. Fast voltage regulators refer to those with response times in the millisecond range, while slow voltage regulators refer to those with action delays in the second range. Response time and action delay parameters for each voltage regulator are extracted, and response time thresholds are set. Devices with response times within 5ms are classified as fast voltage regulators, mainly including SVG devices and STATCOM devices. These devices typically have a rated capacity of 10Mvar or higher and can complete reactive power regulation within milliseconds. Devices with action delays exceeding 2 seconds are classified as slow voltage regulators, mainly including OLTC tap changers and high-voltage capacitor banks. OLTC tap changers typically have 16 adjustment levels and an action delay of approximately 2.5 seconds. Capacitor banks use a group switching method, with each group having a capacity of 2Mvar. This classification based on response speed lays the foundation at the device level for subsequently constructing a dual-time-scale collaborative control architecture.

[0037] S203: Based on the operating status parameters and equipment parameters, set the first time scale corresponding to the fast voltage regulating equipment and the second time scale corresponding to the slow voltage regulating equipment, wherein the first time scale is smaller than the second time scale.

[0038] The first time scale refers to the working time period of the fast voltage regulating equipment, and the second time scale refers to the working time period of the slow voltage regulating equipment.

[0039] Specifically, voltage fluctuation characteristics are analyzed based on the operating status parameters of power grid nodes, and the corresponding operating time scale for each voltage regulating device is determined in conjunction with equipment parameters. For fast voltage regulating devices, based on the frequency of high-frequency voltage fluctuations in the power grid and the device's response capability, a first time scale of 10ms to 100ms is set, corresponding to the fast voltage regulating device performing voltage monitoring and decision-making every 10ms to cope with high-frequency voltage fluctuations such as sudden changes in distributed energy output and instantaneous load impacts. For slow voltage regulating devices, based on the duration of steady-state voltage deviation and the device's action delay characteristics, a second time scale of 2 to 3 seconds is set to match the action delay of the slow device, to correct steady-state voltage deviations caused by prolonged load increases and power grid frequency drift. Through this time scale segmentation, a coordinated voltage regulation mechanism is achieved where fast devices handle high-frequency fluctuations and slow devices correct steady-state deviations, ensuring that the first time scale is smaller than the second time scale, thus ensuring that the two types of devices work complementaryly in different time dimensions.

[0040] Based on the above embodiments, as an optional implementation, in S104, when the voltage data is detected to meet the preset triggering conditions, the slow-scale sub-model in the collaborative decision-making model is enabled to output a slow voltage regulation command. This can be achieved through the following steps S301 or S302.

[0041] S301: When the voltage data is detected to continuously exceed the voltage safety range within the preset sampling period, the slow-scale sub-model in the collaborative decision-making model is activated to output a slow voltage regulation command. The preset sampling period refers to the time interval between voltage monitoring operations performed by the fast-scale sub-model, which can be set to 10 milliseconds in this step. "Continuous exceedance" refers to a state where the voltage data fails to return to the safe voltage range for multiple consecutive sampling periods.

[0042] Specifically, this step implements the triggering and judgment mechanism of the slow-scale sub-model. The fast-scale sub-model samples the grid node voltage every 10 milliseconds, and after each sampling, it determines whether the voltage is within the safe range. When the voltage is detected to be outside the safe range, the fast-scale sub-model first outputs a fast voltage regulation command to drive the fast voltage regulation equipment to adjust. At the same time, a counter is started to record the number of sampling periods in which the voltage continuously exceeds the safe range. For example, when the counter accumulates to 3 sampling periods, that is, when the voltage fails to recover to the safe range within 30 consecutive milliseconds, it is determined that the voltage correction cannot be completed by the fast voltage regulation equipment alone, and the slow-scale sub-model is forcibly triggered. The slow-scale sub-model receives the steady-state average voltage, grid load factor, and the current state of the slow-speed equipment as inputs, performs inference calculations through a multilayer sensor structure, and outputs a slow voltage regulation command to the OLTC tap changer or capacitor bank to perform adjustment, realizing the coordinated cooperation between the fast and slow equipment.

[0043] S302: Calculate the absolute voltage difference between the voltage data and the rated voltage of the grid node. When the absolute voltage difference exceeds the preset difference, the slow-scale sub-model in the collaborative decision-making model is activated to output a slow voltage regulation command.

[0044] The absolute voltage difference refers to the absolute value of the difference between the real-time collected voltage at a power grid node and the rated voltage, used to quantify the degree of voltage deviation. The rated voltage refers to the nominal operating voltage of the power grid node. The preset difference is the voltage deviation threshold that triggers the slow-scale sub-model.

[0045] Specifically, this step determines whether to initiate slow-scale voltage regulation by calculating the absolute value of the voltage deviation. The specific implementation is as follows: The real-time voltage measurement and rated voltage value of the current grid node are obtained, the difference between the two is calculated, and then the absolute value of this difference is taken to obtain the absolute voltage difference. The calculated absolute voltage difference is compared with a preset difference, which is set according to the voltage safety range. For example, for a distribution network with a rated voltage of 10 kV, the preset difference is set to 5% of the rated voltage, i.e., 0.5 kV. When the absolute voltage difference exceeds the preset difference, it indicates that the voltage deviation has reached or exceeded the boundary of the safety range. Relying solely on fast voltage regulation equipment is insufficient to stabilize and correct the voltage to the safety range. At this time, the slow-scale sub-model is activated, outputting a slow-speed voltage regulation command. The slow-scale sub-model determines the adjustment level of the OLTC tap or the number of capacitor banks switched based on the direction and magnitude of the voltage deviation, performing precise steady-state voltage correction.

[0046] Based on the above embodiments, as an optional implementation method, the delayed activation processing of the slow voltage regulating device after S104 can be specifically implemented through the following steps S401-S402.

[0047] S401: During the action delay of the slow voltage regulation device in response to the slow voltage regulation command, predict the expected voltage change based on the slow voltage regulation command; The action delay period refers to the time interval between receiving the slow voltage regulation command and actually completing the voltage regulation action. The expected voltage change refers to the magnitude of the voltage change predicted based on the slow voltage regulation command.

[0048] Specifically, this step predicts voltage changes during the delay period of slow-speed equipment operation. After the slow-scale sub-model outputs a slow-speed voltage regulation command, the slow-speed voltage regulation equipment begins to execute the regulation action. However, due to the inherent mechanical characteristics of the equipment, there is a delay from receiving the command to the action taking effect. Taking an OLTC tap changer as an example, its action delay is approximately 2.5 seconds. During this delay period, the grid voltage is in a transitional state where regulation is not yet complete. The prediction module receives the voltage regulation command output by the slow-scale sub-model and extracts the regulation parameters, such as the OLTC tap changer setting or the number of capacitor banks switched. Based on the regulation parameters, combined with the grid impedance characteristics and the equipment's regulation capability, the expected voltage change is calculated. When the slow-speed voltage regulation command is to increase the OLTC tap changer by one level, based on the tap changer's voltage regulation step size, it is predicted that the voltage will increase by 0.3 kV after the action takes effect; this value is the expected voltage change.

[0049] S402: The fast-scale sub-model controls the output of a compensation voltage regulation command based on the expected voltage change until the slow voltage regulation device takes effect. The compensation voltage regulation command is used to control the fast voltage regulation device to offset voltage fluctuations during the action delay period.

[0050] The compensation voltage regulation command refers to the control signal output by the fast-scale sub-model used to offset voltage fluctuations during the delay period. Offsetting refers to the process of reducing or eliminating the impact of voltage fluctuations by reversing or adjusting in advance.

[0051] This step implements a voltage fluctuation compensation mechanism during the action delay period. The fast-scale sub-model receives the expected voltage change as input and calculates the reactive power required for compensation adjustment based on this change. During the calculation, the expected voltage change is multiplied by a compensation coefficient to obtain the compensation reactive power value. The compensation coefficient is pre-calibrated based on the grid impedance characteristics. The fast-scale sub-model outputs the calculated compensation reactive power value as a compensation voltage regulation command to the fast voltage regulation equipment. After receiving the command, the fast voltage regulation equipment completes its response within milliseconds, adjusting the output reactive power to preemptively pull the voltage change in the expected direction. During the entire slow equipment action delay period, the fast-scale sub-model continuously outputs compensation voltage regulation commands to maintain voltage stability until the slow equipment takes effect. When the OLTC tap changer adjustment is completed and the voltage reaches the target value, the fast-scale sub-model stops outputting compensation voltage regulation commands, and the system returns to the fast-scale dominant mode, completing a complete coordinated voltage regulation process.

[0052] Based on the above embodiments, as an optional implementation method, the adjustable charging amount of the first electric vehicle during the scheduled charging period can be determined through the following steps S501-S503.

[0053] S501: Store voltage regulation events as training samples in the shared experience replay pool. Voltage regulation events are quadruple data including the current state, the action performed, the reward value, and the next state. In this context, a voltage regulation event refers to a complete voltage regulation operation process and its result executed by the collaborative decision-making model during power grid operation. Training samples refer to data units used to train deep neural networks. A shared experience replay pool is a unified database storing training samples for fast-scale and slow-scale sub-models. A four-tuple data structure consists of the current state, the executed action, the reward value, and the next state. The current state includes the voltage value and equipment status; the executed action is the output voltage regulation command; the reward value is the evaluation score calculated based on the voltage regulation effect; and the next state is the voltage value and equipment status after the executed action.

[0054] Specifically, this step involves the acquisition and storage of voltage regulation data. Each time the collaborative decision-making model executes a voltage regulation operation, it records the grid node voltage and voltage regulation equipment status before regulation as the current state, and records the specific voltage regulation command output by the fast-scale or slow-scale sub-model as the execution action. After voltage regulation, it calculates a reward value based on indicators such as whether the voltage returns to a safe range and the voltage regulation response speed, and records the grid node voltage and voltage regulation equipment status after regulation as the next state. These four data items are combined to form a complete four-tuple training sample and written into the shared experience replay pool database. The shared experience replay pool adopts a first-in, first-out storage strategy; when the data volume reaches a preset upper limit, the earliest stored sample is deleted, continuously maintaining a stable database capacity.

[0055] S502: According to the preset alternation frequency, training samples are randomly drawn from the shared experience replay pool in turn, and the fast-scale sub-model and the slow-scale sub-model are trained alternately. Here, "alternating frequency" refers to the switching cycle in which the fast-scale sub-model and the slow-scale sub-model are trained sequentially. "Random sampling" refers to the process of randomly selecting training samples from the shared experience replay pool. "Alternating training" refers to a training method in which the fast-scale sub-model and the slow-scale sub-model update their parameters alternately at a set frequency.

[0056] Specifically, this step implements an alternating training mechanism for the two-scale sub-models. The training process controls the training order of the two sub-models according to a preset alternation frequency. For example, the alternation frequency is set to switch training objects after every 100 iterations. At the start of training, the fast-scale sub-model is trained first. 32 training samples are randomly selected from the shared experience replay pool to form a batch, and this batch data is input into the fast-scale sub-model for forward and backward propagation. After the fast-scale sub-model completes 100 iterations, its training process is paused, and the slow-scale sub-model is trained instead, again using 32 randomly selected training samples from the shared experience replay pool. The two sub-models are trained alternately, and the samples in the shared experience replay pool are reused by both sub-models, improving data utilization efficiency. The random sampling mechanism breaks the temporal correlation between samples, preventing the model training from getting trapped in local optima.

[0057] S503: Minimize the loss function between the predicted value and the target value, and update the deep neural network parameters of the collaborative decision-making model using the gradient descent algorithm until the collaborative decision-making model converges.

[0058] In this context, the loss function is a mathematical function that measures the difference between the predicted value and the target value of a deep neural network. The predicted value is the estimated reward value calculated by the deep neural network based on the current state and the action performed. The target value is the actual reward value recorded in the training samples. Gradient descent is an optimization method that calculates the gradient of the loss function with respect to the network parameters and adjusts the parameters in the opposite direction of the gradient. Deep neural network parameters include the connection weights and biases between neurons in each layer. Convergence refers to the state where, after multiple iterations of training, the loss function value tends to stabilize and no longer decreases significantly.

[0059] Specifically, this step implements the parameter optimization process of the collaborative decision-making model. During training, the current state and actions in the training samples are input into the deep neural network for forward computation, and the network output layer obtains the predicted value. The predicted value is compared with the target value in the training samples, the difference between the two is calculated, and the square is taken as the single-sample loss value. The loss values ​​of all samples in the batch are averaged to obtain the batch loss function value. The gradient descent algorithm calculates the partial derivative of the loss function with respect to each network parameter, obtaining the gradient value of each parameter. The parameter adjustment is calculated by multiplying the learning rate by the gradient value, and each parameter is updated along the negative gradient direction. The fast-scale sub-model and the slow-scale sub-model execute the above parameter update process respectively. Iterative training is continuously performed, and the change in the loss function value is monitored. When the change in the loss function value after 100 consecutive iterations is less than a preset threshold, the collaborative decision-making model is determined to have reached convergence, and training is completed.

[0060] The reward value is calculated using a reward function, which is: R=α×(1-|V_real-V_std| / V_std)-β×N-γ×T; Where R represents the reward value of the reward function, V_real represents the instantaneous voltage value of the grid node in the voltage data, V_std represents the rated voltage, N represents the number of actions of the fast voltage regulating device or the slow voltage regulating device, T represents the voltage over-limit duration, α represents the voltage control accuracy weight coefficient, β represents the device action penalty weight coefficient, and γ represents the voltage over-limit penalty weight coefficient.

[0061] The reward value R represents the evaluation score obtained by the collaborative decision-making model after performing voltage regulation operations; a higher value indicates a better voltage regulation effect. The instantaneous voltage value V_real represents the actual voltage value measured at a grid node at a certain moment. The rated voltage V_std represents the nominal operating voltage value of the grid node. The number of equipment actions N represents the cumulative number of regulation operations performed by fast or slow voltage regulating equipment in a single voltage regulation event. The voltage exceedance duration T represents the duration for which the voltage exceeds the safe range. The voltage control accuracy weighting coefficient α is used to adjust the degree of influence of voltage deviation on the reward value. The equipment action penalty weighting coefficient β is used to adjust the degree of influence of the number of equipment actions on the reward value. The voltage exceedance penalty weighting coefficient γ is used to adjust the degree of influence of the voltage exceedance duration on the reward value.

[0062] The reward function consists of three parts. The first part calculates the voltage control accuracy reward. First, the difference between the instantaneous voltage value and the rated voltage is calculated and its absolute value is taken. Then, this is divided by the rated voltage to obtain the voltage deviation rate. Subtracting the voltage deviation rate from 1 gives the voltage control accuracy, which is then multiplied by a weighting coefficient α to obtain the voltage control accuracy reward. The second part calculates the equipment action penalty. The number of equipment actions is multiplied by a weighting coefficient β and taken as a negative value; the more actions the equipment takes, the greater the penalty. The third part calculates the voltage limit violation penalty. The duration of the voltage limit violation is multiplied by a weighting coefficient γ and taken as a negative value; the longer the duration of the voltage limit violation, the greater the penalty. The three parts are added together to obtain the final reward value. A larger reward value is obtained when the voltage control accuracy is high, the number of equipment actions is low, and the duration of the voltage limit violation is short.

[0063] This reward function guides the collaborative decision-making model to learn a voltage regulation strategy that balances voltage control quality and equipment lifespan. A voltage control accuracy reward incentivizes the model to precisely control the voltage near the rated level, improving voltage control quality. An equipment action penalty inhibits the model from frequently calling voltage regulation devices, reducing mechanical wear and extending equipment lifespan. A voltage exceedance penalty prompts the model to respond quickly to voltage anomalies, shortening the duration of voltage exceeding safe limits and ensuring safe grid operation. The setting of three weighting coefficients balances the importance of different optimization objectives, enabling the collaborative decision-making model to gradually converge to the voltage regulation control strategy with optimal overall performance during deep reinforcement learning training, achieving multi-objective collaborative optimization of voltage quality, equipment lifespan, and response speed.

[0064] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of the application.

[0065] Please see Figure 2 This illustration shows a schematic diagram of a dual-timescale DRL cooperative voltage regulation control system provided in an exemplary embodiment of this application. The system can be implemented as all or part of a system through software, hardware, or a combination of both. The dual-timescale DRL cooperative voltage regulation control system includes: The model building module is used to build a collaborative decision-making model for dual-timescale DRL. The collaborative decision-making model includes a fast-scale sub-model and a slow-scale sub-model that share an experience replay pool. The voltage acquisition module is used to collect voltage data from power grid nodes in real time and input the voltage data into the collaborative decision-making model; The fast voltage regulation module is used to monitor voltage data routinely using the fast-scale sub-model in the collaborative decision-making model, and outputs a fast voltage regulation command when the voltage data exceeds the preset voltage safety range. The fast voltage regulation command is used to control the fast voltage regulation equipment to regulate the grid voltage. The slow voltage regulation module is used to activate the slow-scale sub-model in the collaborative decision-making model to output a slow voltage regulation command when the monitored voltage data meets the preset trigger conditions, until the adjusted target voltage data is restored to the voltage safety range. The slow voltage regulation command is used to control the slow voltage regulation equipment to regulate the grid voltage. The model update module is used to store voltage regulation events as training samples in the shared experience replay pool, and randomly select training samples from the shared experience replay pool to update the collaborative decision-making model until the collaborative decision-making model converges.

[0066] This application also provides a computer storage medium that can store multiple instructions. The instructions are adapted to be loaded by a processor and executed as described in the above embodiments of the dual-timescale DRL coordinated voltage regulation control method. For the specific execution process, please refer to the detailed description of the embodiments, which will not be repeated here.

[0067] Please see Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown, the electronic device 300 may include: at least one processor 301, at least one network interface 304, user interface 303, memory 305, and at least one communication bus 302.

[0068] The communication bus 302 is used to enable communication between these components.

[0069] The user interface 303 may include a display screen and a camera.

[0070] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0071] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling data stored in the memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of digital signal processing, field-programmable gate array, or programmable logic array. The processor 301 may integrate one or more of the following: a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0072] The memory 305 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 305 may include a non-transitory computer-readable medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. Figure 3 As shown, the memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a dual-time-scale DRL coordinated voltage regulation control method.

[0073] exist Figure 3 In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 301 can be used to call an application program stored in the memory 305 for a dual time scale DRL cooperative voltage regulation control method. When executed by one or more processors, the electronic device executes one or more methods as described in the above embodiments.

[0074] An electronic device readable storage medium stores instructions that, when executed by one or more processors, cause the electronic device to perform one or more methods as described in the above embodiments.

[0075] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0076] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0077] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0078] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0079] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0080] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0081] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and practical application disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure.

Claims

1. A dual time scale (DRL) coordinated voltage regulation control method, characterized in that, The method comprises: constructing a collaborative decision model of a double-time-scale DRL, the collaborative decision model comprising a fast-scale submodel and a slow-scale submodel sharing an experience replay pool; collecting voltage data of a power grid node in real time and inputting the voltage data into the collaborative decision model; normalizing monitoring of the voltage data using the fast-scale submodel in the collaborative decision model, and outputting a fast voltage regulation instruction for controlling a fast voltage regulation device to regulate voltage of the power grid when the voltage data exceeds a preset voltage safety range; when it is monitored that the voltage data meets a preset triggering condition, enabling the slow-scale submodel in the collaborative decision model to output a slow voltage regulation instruction until target voltage data after regulation returns to the voltage safety range, the slow voltage regulation instruction being used to control a slow voltage regulation device to regulate voltage of the power grid; storing a voltage regulation event as a training sample in the experience replay pool, and updating the collaborative decision model by randomly extracting the training sample from the experience replay pool until the collaborative decision model converges.

2. The method of claim 1, wherein, Before the constructing a collaborative decision model of a double-time-scale DRL, the collaborative decision model comprising a fast-scale submodel and a slow-scale submodel sharing an experience replay pool, further comprising: obtaining operating state parameters of a power grid node and device parameters of multiple voltage regulation devices; dividing the multiple voltage regulation devices into fast voltage regulation devices and slow voltage regulation devices according to the device parameters; setting a first time scale corresponding to the fast voltage regulation devices and a second time scale corresponding to the slow voltage regulation devices according to the operating state parameters and the device parameters, the first time scale being smaller than the second time scale.

3. The method of claim 1, wherein, The when it is monitored that the voltage data meets a preset triggering condition, enabling the slow-scale submodel in the collaborative decision model to output a slow voltage regulation instruction, comprises: when it is monitored that the voltage data continuously exceeds the voltage safety range within a preset sampling period, enabling the slow-scale submodel in the collaborative decision model to output a slow voltage regulation instruction; or, calculating an absolute voltage difference between the voltage data and a rated voltage of the power grid node, and when the absolute voltage difference exceeds a preset difference value, enabling the slow-scale submodel in the collaborative decision model to output a slow voltage regulation instruction.

4. The method of claim 1, wherein, After the when it is monitored that the voltage data meets a preset triggering condition, enabling the slow-scale submodel in the collaborative decision model to output a slow voltage regulation instruction, further comprising: during an action delay period of the slow voltage regulation device in response to the slow voltage regulation instruction, predicting an expected voltage change amount based on the slow voltage regulation instruction; controlling the fast-scale submodel to output a compensation voltage regulation instruction according to the expected voltage change amount until the action of the slow voltage regulation device takes effect, the compensation voltage regulation instruction being used to control the fast voltage regulation device to offset voltage fluctuation during the action delay period.

5. The method of claim 1, wherein, The storing a voltage regulation event as a training sample in the experience replay pool, and updating the collaborative decision model by randomly extracting the training sample from the experience replay pool until the collaborative decision model converges, comprises: store voltage regulation events as training samples into the shared experience replay pool, the voltage regulation events being four-tuple data including a current state, an executed action, a reward value and a next state; alternately train the fast-scale sub-model and the slow-scale sub-model by sequentially and randomly sampling training samples from the shared experience replay pool according to a preset alternating frequency; update the deep neural network parameters of the collaborative decision model by a gradient descent algorithm until the collaborative decision model converges, with a loss function between a predicted value and a target value being minimized.

6. The method of claim 5, wherein, The reward value is calculated by a reward function, and the reward function is: R = α × (1 - |V real - V std| / V std) - β × N - γ × T; wherein R represents the reward value of the reward function, V real represents an instantaneous voltage value of a power grid node in the voltage data, V std represents a rated voltage, N represents a number of actions of the fast voltage regulating device or the slow voltage regulating device, T represents a voltage out-of-limit duration, α represents a voltage control accuracy weight coefficient, β represents a device action penalty weight coefficient, and γ represents a voltage out-of-limit penalty weight coefficient.

7. The method of claim 1, wherein, The fast-scale sub-model is a CNN structure model, and the slow-scale sub-model is an MLP structure model.

8. A dual time scale (DRL) coordinated voltage control system, comprising: The system comprises: a model construction module configured to construct a collaborative decision model of a double-time-scale DRL, the collaborative decision model comprising a fast-scale sub-model and a slow-scale sub-model of a shared experience replay pool; a voltage acquisition module configured to acquire voltage data of a power grid node in real time and input the voltage data into the collaborative decision model; a fast voltage regulating module configured to use the fast-scale sub-model in the collaborative decision model to perform normal monitoring on the voltage data, and output a fast voltage regulating instruction for controlling a fast voltage regulating device to regulate a power grid voltage when the voltage data exceeds a preset voltage safety range; a slow voltage regulating module configured to enable the slow-scale sub-model in the collaborative decision model to output a slow voltage regulating instruction when it is monitored that the voltage data meets a preset triggering condition, until a target voltage data after regulation returns to the voltage safety range, the slow voltage regulating instruction being used to control a slow voltage regulating device to regulate a power grid voltage; a model update module configured to store voltage regulation events as training samples into the shared experience replay pool, and update the collaborative decision model by randomly sampling the training samples from the shared experience replay pool until the collaborative decision model converges.

9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions adapted to be loaded and executed by a processor to perform the method of any one of claims 1-7.

10. An electronic device, comprising: The electronic device comprises a processor, a memory and a transceiver, the memory is configured to store instructions, the transceiver is configured to communicate with other devices, and the processor is configured to execute the instructions stored in the memory to enable the electronic device to perform the method of any one of claims 1-7.

Images