Station terminal control method and device, station terminal and equipment

Abstract

The invention relates to the technical field of power supply and distribution, in particular to a station terminal control method and device, a station terminal and equipment, and the method comprises the steps: obtaining a target station terminal control model obtained through pre-training and a current station terminal state of the station terminal; based on the target station terminal control model, determining a target control action corresponding to a target action value, the target action value being a maximum action value corresponding to the current station terminal state; and the terminal of the control station executes the target control action. According to the scheme, the station terminal executes the target control action without intervention of a cloud end or a man-machine interaction terminal, and the state of the station terminal can meet the requirement of normal work after the target control action is executed, so that the control efficiency of the station terminal is improved, and the control cost is reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of power supply and distribution, and in particular to a station terminal control method, device, station terminal and equipment. Background Art

[0002] In recent years, with the continuous development of new energy technologies, the construction and management of power supply and distribution systems serving new energy equipment have become increasingly important. Among them, the coordinated control of the power supply and distribution system faces problems such as voltage over-limit and reverse overload caused by the large-scale access of distributed photovoltaics, resulting in huge challenges in the control and monitoring of the power supply and distribution system.

[0003] In order to solve this problem, station terminals (Data Transfer Unit, DTU) have begun to be widely used in power supply and distribution systems. Station terminals are generally installed in conventional switchgear (stations), outdoor small switchgear, ring network cabinets, small substations, box-type substations, etc. By collecting, transmitting and receiving real-time data of the power supply and distribution system through station terminals, real-time monitoring and control of the operation of the power supply and distribution system can be achieved.

[0004] However, the applicant found that in the related technology, it is necessary for the cloud in the power supply and distribution system to send control instructions to the station terminal, or for the corresponding technical personnel to send control instructions to the station terminal through the human-computer interaction terminal in the power supply and distribution system, so as to instruct the station terminal to perform corresponding control actions based on the received control instructions. In the above scheme, more communication bandwidth is required, the control efficiency is low, and the corresponding cloud or human-computer interaction terminal needs to be configured in the power supply and distribution system, resulting in high costs. Summary of the invention

[0005] In order to solve the problems in the related art, the embodiments of the present disclosure provide a station terminal control method, device, station terminal and equipment.

[0006] In a first aspect, an embodiment of the present disclosure provides a station terminal control method, including:

[0007] Obtaining a pre-trained target station terminal control model and a current station terminal state of the station terminal, wherein the target station terminal control model includes an action value function value table, and the action value function value table is used to indicate the corresponding relationship between the station terminal state, the control action, and the action value;

[0008] Based on the target station terminal control model, determine the target control action corresponding to the target action value, where the target action value is the maximum action value corresponding to the current station terminal state;

[0009] The terminal at the control station executes the target control action.

[0010] In one implementation of the present disclosure, before obtaining the pre-trained target station terminal control model, the method further includes:

[0011] Obtaining an initial station terminal control model;

[0012] Acquire a first station terminal state of the current station terminal, and determine a first control action corresponding to a first action value based on a station terminal control model, wherein the first action value is a maximum action value corresponding to the first station terminal state, and the station terminal control model includes an action value function value table, and the action value function value table is used to indicate a corresponding relationship between the station terminal state, the control action, and the action value;

[0013] Acquire a second station terminal state of the station terminal after the station terminal executes the first control action and a distribution network parameter detected by the station terminal;

[0014] Obtaining a second action value based on the station terminal control model, wherein the second action value is a maximum action value corresponding to the second station terminal state;

[0015] Acquire an updated action value based on the first control action, the distribution network parameter, the first action value, and the second action value, and update the first action value in the action value function value table based on the updated action value;

[0016] In response to the action-value function value table converging, the station terminal control model is determined as the target station terminal control model.

[0017] In one implementation of the present disclosure, obtaining an update action value based on the first control action, the distribution network parameter, the first action value, and the second action value includes:

[0018] Obtaining a real-time reward R(s,a) based on the first control action and the distribution network parameters;

[0019] Based on Q g (s,a)=Q1(s,a)+α[R(s,a)+γmaxQ(s′,a′)-Q1(s,a)] to get the updated action value Q g (s,a), where Q1(s,a) is the first action value, s is the terminal state of the first station, a is the first control action, α is a preset learning rate parameter, γ is a preset conversion factor, maxQ(s′,a′) is the second action value, s′ is the terminal state of the second station, and a′ is the second control action.

[0020] In an implementation of the present disclosure, the distribution network parameter includes at least one of a current value, a voltage value, current waveform data within a preset time length, and voltage waveform data within a preset time length.

[0021] In one implementation of the present disclosure, obtaining a real-time reward R(s,a) based on the first control action and the distribution network parameters includes:

[0022] Based on the first control action, the current value in the distribution network parameters and the current waveform data within the preset time, a current real-time reward R1(s,a) is obtained, and based on the first control action, the voltage value in the distribution network parameters and the voltage waveform data within the preset time, a voltage real-time reward R2(s,a) is obtained;

[0023] The real-time reward R(s,a) is obtained based on R(s,a)=k1R1(s,a)+k2R2(s,a), where k1 is the preset current reward weight and k2 is the preset voltage reward weight.

[0024] In one implementation of the present disclosure, the control action includes at least one of no action, closing, and opening;

[0025] And / or, the station terminal status includes at least one of no fault, single-phase ground short circuit fault, two-phase short circuit fault, two-phase ground short circuit fault, and three-phase short circuit fault.

[0026] In a second aspect, an embodiment of the present disclosure provides a station terminal control device, including:

[0027] A state acquisition module is configured to acquire a pre-trained target station terminal control model and a current station terminal state of the station terminal, wherein the target station terminal control model includes an action value function value table, and the action value function value table is used to indicate the corresponding relationship between the station terminal state, the control action and the action value;

[0028] An action acquisition module is configured to determine a target control action corresponding to a target action value based on a target station terminal control model, wherein the target action value is a maximum action value corresponding to a current station terminal state;

[0029] The station terminal control module is configured to control the station terminal to perform target control actions.

[0030] In one implementation of the present disclosure, the status acquisition module is specifically configured as follows:

[0031] Obtaining an initial station terminal control model;

[0032] Acquire a first station terminal state of the current station terminal, and determine a first control action corresponding to a first action value based on a station terminal control model, wherein the first action value is a maximum action value corresponding to the first station terminal state, and the station terminal control model includes an action value function value table, and the action value function value table is used to indicate a corresponding relationship between the station terminal state, the control action, and the action value;

[0033] Acquire a second station terminal state of the station terminal after the station terminal executes the first control action and a distribution network parameter detected by the station terminal;

[0034] Obtaining a second action value based on the station terminal control model, wherein the second action value is a maximum action value corresponding to the second station terminal state;

[0035] Acquire an updated action value based on the first control action, the distribution network parameter, the first action value, and the second action value, and update the first action value in the action value function value table based on the updated action value;

[0036] In response to the action-value function value table converging, the station terminal control model is determined as the target station terminal control model.

[0037] In one implementation of the present disclosure, the status acquisition module is specifically configured as follows:

[0038] Obtaining a real-time reward R(s,a) based on the first control action and the distribution network parameters;

[0039] Based on Q g (s,a)=Q1(s,a)+α[R(s,a)+γmaxQ(s′,a′)-Q1(s,a)] to get the updated action value Q g (s,a), where Q1(s,a) is the first action value, s is the terminal state of the first station, a is the first control action, α is a preset learning rate parameter, γ is a preset conversion factor, maxQ(s′,a′) is the second action value, s′ is the terminal state of the second station, and a′ is the second control action.

[0040] In an implementation of the present disclosure, the distribution network parameter includes at least one of a current value, a voltage value, current waveform data within a preset time length, and voltage waveform data within a preset time length.

[0041] In one implementation of the present disclosure, the status acquisition module is specifically configured as follows:

[0042] Based on the first control action, the current value in the distribution network parameters and the current waveform data within the preset time, a current real-time reward R1(s,a) is obtained, and based on the first control action, the voltage value in the distribution network parameters and the voltage waveform data within the preset time, a voltage real-time reward R2(s,a) is obtained;

[0043] The real-time reward R(s,a) is obtained based on R(s,a)=k1R1(s,a)+k2R2(s,a), where k1 is the preset current reward weight and k2 is the preset voltage reward weight.

[0044] In one implementation of the present disclosure, the control action includes at least one of no action, closing, and opening;

[0045] And / or, the station terminal status includes at least one of no fault, single-phase ground short circuit fault, two-phase short circuit fault, two-phase ground short circuit fault, and three-phase short circuit fault.

[0046] In a third aspect, a station terminal is provided in an embodiment of the present disclosure, characterized in that the station terminal includes the station terminal control device of any one of the second aspects.

[0047] In one implementation of the present disclosure, the station terminal is further used for:

[0048] Collect terminal working data and distribution network data;

[0049] Send terminal working data and distribution network data to the distribution automation master station;

[0050] The receiving distribution automation master station determines the station terminal control model based on the collected terminal working data and distribution network data.

[0051] In a fourth aspect, an embodiment of the present disclosure provides a distribution automation master station, which is used for:

[0052] The terminal working data and distribution network data uploaded by the terminal at the receiving station;

[0053] The station terminal control model is determined based on the terminal working data and the distribution network data, and the station terminal control model is sent to the station terminal.

[0054] In a fifth aspect, a distribution network control system is provided in an embodiment of the present disclosure, characterized in that it includes the station terminal of any one of the third aspect and the distribution automation master station of the fourth aspect.

[0055] In a sixth aspect, an embodiment of the present disclosure provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first aspects.

[0056] According to the technical solution provided by the embodiment of the present disclosure, by obtaining the pre-trained target station terminal control model and the current station terminal state of the station terminal; based on the target station terminal control model, determine the target control action corresponding to the target action value, wherein the target action value is the maximum action value corresponding to the current station terminal state; control the station terminal to execute the target control action. This solution enables the station terminal to execute the target control action that matches the maximum action value corresponding to the current station terminal state without the intervention of the cloud or human-computer interaction terminal, thereby ensuring that the station terminal state of the station terminal after executing the target control action can meet the requirements of normal operation. Therefore, this solution improves the control efficiency of the station terminal and reduces the control cost.

[0057] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0059] Figure 1 A flowchart of a station terminal control method according to an embodiment of the present disclosure is shown.

[0060] Figure 2 A structural block diagram of a station terminal control device according to an embodiment of the present disclosure is shown.

[0061] Figure 3 A structural block diagram of a station terminal according to an embodiment of the present disclosure is shown.

[0062] Figure 4 A structural block diagram of a power distribution network control system according to an embodiment of the present disclosure is shown.

[0063] Figure 5 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0064] Figure 6 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0065] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0066] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.

[0067] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0068] In the present disclosure, if it involves operations of obtaining user information or user data or displaying user information or user data to others, the operations are all authorized and confirmed by the user, or actively selected by the user.

[0069] In the power supply and distribution system, the station terminal is generally installed in conventional switchgear (station), outdoor small switchgear, ring network cabinet, small substation, box-type substation, etc. The station terminal collects, transmits and receives real-time data of the power supply and distribution system, which can realize real-time monitoring and control of the operation of the power supply and distribution system.

[0070] The applicant found that in the related art, it is necessary for the cloud in the power supply and distribution system to send control instructions to the station terminal, or for the corresponding technical personnel to send control instructions to the station terminal through the human-computer interaction terminal in the power supply and distribution system, so as to instruct the station terminal to perform the corresponding control action based on the received control instruction. In the above scheme, more communication bandwidth is required, the control efficiency is low, and the corresponding cloud or human-computer interaction terminal needs to be configured in the power supply and distribution system, resulting in high cost.

[0071] In order to solve the above problems, the embodiments of the present disclosure provide a station terminal control method, device, station terminal, master station, system and equipment.

[0072] According to the technical solution provided by the embodiment of the present disclosure, by obtaining the pre-trained target station terminal control model and the current station terminal state of the station terminal; based on the target station terminal control model, determine the target control action corresponding to the target action value, wherein the target action value is the maximum action value corresponding to the current station terminal state; control the station terminal to execute the target control action. This solution enables the station terminal to execute the target control action that matches the maximum action value corresponding to the current station terminal state without the intervention of the cloud or human-computer interaction terminal, thereby ensuring that the station terminal state of the station terminal after executing the target control action can meet the requirements of normal operation. Therefore, this solution improves the control efficiency of the station terminal and reduces the control cost.

[0073] Figure 1FIG. 1 is a flow chart showing a method for controlling a station terminal according to an embodiment of the present disclosure. Figure 1 As shown, the station terminal control method includes the following steps S101-S103:

[0074] In step S101, a pre-trained target station terminal control model and a current station terminal state of the station terminal are obtained;

[0075] The target station terminal control model includes an action value function value table, which is used to indicate the correspondence between the station terminal state, control action and action value.

[0076] In step S102, based on the terminal control model of the target station, a target control action corresponding to the target action value is determined;

[0077] The target action value is the maximum action value corresponding to the current station terminal state.

[0078] In step S103, the control station terminal performs the target control action.

[0079] In one implementation of the present disclosure, the distribution network includes a distribution automation master station, station terminals, switch stations, power quality monitoring devices, fault monitoring devices, etc., wherein the station terminals communicate with the distribution automation master station upward and communicate with the switch stations, power quality monitoring devices, fault monitoring devices, etc. downward through wired or wireless communication, collect and process the obtained data, and perform processing and decision-making to realize the opening and closing of switches, isolate non-fault areas, and provide technical support for the overall distribution network safety. The distribution automation master station is responsible for centralizing the data of each station terminal, and performing model learning, and sending the corresponding model to the station terminal as needed.

[0080] In one implementation of the present disclosure, the control action includes at least one of no action, closing, and opening;

[0081] And / or, the station terminal status includes at least one of no fault, single-phase ground short circuit fault, two-phase short circuit fault, two-phase ground short circuit fault, and three-phase short circuit fault.

[0082] In one implementation of the present disclosure, the action value can be understood as indicating the degree of improvement of the state of the station terminal, or the degree of improvement of the state of the station terminal and the distribution network, after the station terminal performs the corresponding operation action. For example, if the action value corresponding to a certain operation action is greater, the station terminal will improve the current state of the station terminal more after performing the operation action; conversely, if the action value corresponding to a certain operation action is smaller, the station terminal will improve the current state of the station terminal less after performing the operation action.

[0083] According to the technical solution provided by the embodiment of the present disclosure, by obtaining the pre-trained target station terminal control model and the current station terminal state of the station terminal; based on the target station terminal control model, determine the target control action corresponding to the target action value, wherein the target action value is the maximum action value corresponding to the current station terminal state; control the station terminal to execute the target control action. This solution enables the station terminal to execute the target control action that matches the maximum action value corresponding to the current station terminal state without the intervention of the cloud or human-computer interaction terminal, thereby ensuring that the station terminal state of the station terminal after executing the target control action can meet the requirements of normal operation. Therefore, this solution improves the control efficiency of the station terminal and reduces the control cost.

[0084] In one embodiment of the present disclosure, before obtaining the pre-trained target station terminal control model, the method further includes:

[0085] Obtaining an initial station terminal control model;

[0086] Acquire a first station terminal state of the current station terminal, and determine a first control action corresponding to a first action value based on a station terminal control model, wherein the first action value is a maximum action value corresponding to the first station terminal state, and the station terminal control model includes an action value function value table, and the action value function value table is used to indicate a corresponding relationship between the station terminal state, the control action, and the action value;

[0087] Acquire a second station terminal state of the station terminal after the station terminal executes the first control action and a distribution network parameter detected by the station terminal;

[0088] Obtaining a second action value based on the station terminal control model, wherein the second action value is a maximum action value corresponding to the second station terminal state;

[0089] Acquire an updated action value based on the first control action, the distribution network parameter, the first action value, and the second action value, and update the first action value in the action value function value table based on the updated action value;

[0090] In response to the action-value function value table converging, the station terminal control model is determined as the target station terminal control model.

[0091] In one implementation of the present disclosure, determining the first control action corresponding to the first action value based on the station terminal control model can be understood as querying at least one action value corresponding to the current station terminal state in the action value function value table, and determining the control action corresponding to the largest action value among the at least one action value as the first control action.

[0092] In an implementation of the present disclosure, the distribution network parameter includes at least one of a current value, a voltage value, current waveform data within a preset time length, and voltage waveform data within a preset time length.

[0093] In an implementation of the present disclosure, the update action value may be understood as indicating the degree of improvement of the state of the station terminal and the state of the distribution network after the station terminal executes the first control action.

[0094] In one implementation of the present disclosure, obtaining an updated action value based on the first control action, the distribution network parameters, the first action value, and the second action value can be understood as substituting the first control action, the distribution network parameters, the first action value, and the second action value for calculation based on a pre-acquired algorithm to obtain the updated action value; or, obtaining an updated action value based on the first control action, the distribution network parameters, the first action value, and the second action value can also be understood as obtaining a pre-generated action value update table, and searching for the corresponding updated action value in the action value update table based on the first control action, the distribution network parameters, the first action value, and the second action value.

[0095] According to the technical solution provided by the embodiment of the present disclosure, by obtaining the second station terminal state of the station terminal after the station terminal executes the first control action and the distribution network parameters detected by the station terminal; obtaining the second action value based on the initial station terminal control model, wherein the second action value is the maximum action value corresponding to the second station terminal state; obtaining the updated action value based on the first control action, the distribution network parameters, the first action value, and the second action value, and updating the first action value in the action value function value table based on the updated action value; in response to the convergence of the action value function value table, determining the station terminal control model as the target station terminal control model, it can be ensured that the action value function value table in the acquired target station terminal control model can accurately reflect the correspondence between the station terminal state, the control action and the action value, which helps to improve the control efficiency of the station terminal and reduce the control cost of the station terminal.

[0096] In one embodiment of the present disclosure, the update action value is obtained based on the first control action, the distribution network parameter, the first action value, and the second action value, which can be obtained in the following manner:

[0097] Obtaining a real-time reward R(s,a) based on the first control action and the distribution network parameters;

[0098] Based on Q g (s,a)=Q1(s,a)+α[R(s,a)+γmaxQ(s′,a′)-Q1(s,a)] to get the updated action value Q g(s,a), where Q1(s,a) is the first action value, s is the terminal state of the first station, a is the first control action, α is a preset learning rate parameter, γ is a preset conversion factor, maxQ(s′,a′) is the second action value, s′ is the terminal state of the second station, and a′ is the second control action.

[0099] In an implementation of the present disclosure, the value range of the learning rate parameter may be 0<α≤1, and the value range of the conversion factor may be 0≤γ≤1.

[0100] In one implementation of the present disclosure, obtaining a real-time reward based on the first control action and the distribution network parameters can be understood as substituting the first control action and the distribution network parameters into a pre-acquired algorithm for calculation to obtain the real-time reward, wherein the algorithm may include a variety of commonly used algorithms in the industry, such as Monte Carlo algorithm, Bayesian algorithm, etc.; or, it can also be understood as obtaining a pre-generated real-time reward table, and searching the real-time reward corresponding to the first control action and the distribution network parameters in the real-time reward table based on the first control action and the distribution network parameters.

[0101] According to the technical solution provided by the embodiment of the present disclosure, by obtaining the update action value through the above solution, the accuracy of the obtained update action value can be improved, which helps to improve the control efficiency of the station terminal and reduce the control cost of the station terminal.

[0102] In one embodiment of the present disclosure, obtaining a real-time reward R(s,a) based on the first control action and the distribution network parameters includes:

[0103] Based on the first control action, the current value in the distribution network parameters and the current waveform data within the preset time, a current real-time reward R1(s,a) is obtained, and based on the first control action, the voltage value in the distribution network parameters and the voltage waveform data within the preset time, a voltage real-time reward R2(s,a) is obtained;

[0104] The real-time reward R(s,a) is obtained based on R(s,a)=k1R1(s,a)+k2R2(s,a), where k1 is the preset current reward weight and k2 is the preset voltage reward weight.

[0105] In an implementation of the present disclosure, the values ​​of k1 and k2 may both be 0.5.

[0106] In one implementation of the present disclosure, obtaining a real-time current reward based on the first control action, the current value in the distribution network parameters, and the current waveform data within a preset time period can be understood as substituting the current value in the distribution network parameters and the current waveform data within a preset time period for calculation based on a pre-acquired algorithm to obtain the real-time current reward, wherein the algorithm may include a variety of algorithms commonly used in the industry, such as Monte Carlo algorithm, Bayesian algorithm, etc.; or, it can also be understood as obtaining a pre-generated current reward table, and searching the current reward table for the corresponding real-time current reward based on the first control action, the current value in the distribution network parameters, and the current waveform data within a preset time period.

[0107] Exemplarily, based on the current value in the distribution network parameters and the current waveform data within a preset time length, the status of the current station terminal can be determined, where the status of the station terminal includes "S1: No fault", "S2: Single-phase ground short circuit fault", "S3: Two-phase short circuit fault", "S4: Two-phase ground short circuit fault", and "S5: Three-phase short circuit fault".

[0108] If the current state of the terminal is determined to be "S1: no fault" and the first control action is "no action", the real-time current reward is 200; if the current state of the terminal is determined to be "S1: no fault" and the first control action is "a2: open", the real-time current reward is -1; if the current state of the terminal is determined to be "S1: no fault" and the first control action is "a3: closed", the real-time current reward is 100.

[0109] If the current state of the terminal is determined to be "S2: single-phase ground short-circuit fault" and the first control action is "no action", the real-time current reward is -1; if the current state of the terminal is determined to be "S2: single-phase ground short-circuit fault" and the first control action is "a2: open", the real-time current reward is 200; if the current state of the terminal is determined to be "S2: single-phase ground short-circuit fault" and the first control action is "a3: close", the real-time current reward is -1.

[0110] If the current state of the terminal is determined to be "S3: two-phase short circuit fault" and the first control action is "no action", the real-time current reward is -1; if the current state of the terminal is determined to be "S3: two-phase short circuit fault" and the first control action is "a2: open", the real-time current reward is 200; if the current state of the terminal is determined to be "S3: two-phase short circuit fault" and the first control action is "a3: close", the real-time current reward is -1.

[0111] If the current state of the terminal is determined to be "S4: two-phase ground short circuit fault" and the first control action is "no action", the real-time current reward is -1; if the current state of the terminal is determined to be "S4: two-phase ground short circuit fault" and the first control action is "a2: open", the real-time current reward is 200; if the current state of the terminal is determined to be "S4: two-phase ground short circuit fault" and the first control action is "a3: close", the real-time current reward is -1.

[0112] If the current state of the station terminal is determined to be "S5: three-phase short circuit fault" and the first control action is "no action", the real-time current reward is -1; if the current state of the station terminal is determined to be "S5: three-phase short circuit fault" and the first control action is "a2: open", the real-time current reward is 200; if the current state of the station terminal is determined to be "S5: three-phase short circuit fault" and the first control action is "a3: close", the real-time current reward is -1.

[0113] In one implementation of the present disclosure, obtaining a real-time voltage reward based on the first control action, the voltage value in the distribution network parameters, and the voltage waveform data within a preset time period can be understood as substituting the voltage value in the distribution network parameters and the voltage waveform data within a preset time period into a calculation based on a pre-acquired algorithm to obtain the real-time voltage reward, wherein the algorithm may include a variety of algorithms commonly used in the industry, such as Monte Carlo algorithm, Bayesian algorithm, etc.; or, it can also be understood as obtaining a pre-generated voltage reward table, and searching the voltage reward table for the corresponding real-time voltage reward based on the first control action, the voltage value in the distribution network parameters, and the voltage waveform data within a preset time period.

[0114] Exemplarily, based on the voltage value in the distribution network parameters and the voltage waveform data within a preset time period, the state of the current station terminal can be determined.

[0115] If the current state of the station terminal is determined to be "S1: no fault" and the first control action is "no action", the real-time voltage reward is 100; if the current state of the station terminal is determined to be "S1: no fault" and the first control action is "a2: open", the real-time voltage reward is -1; if the current state of the station terminal is determined to be "S1: no fault" and the first control action is "a3: closed", the real-time voltage reward is 50.

[0116] If the state of the current station terminal is determined to be "S2: single-phase ground short-circuit fault" and the first control action is "no action", the real-time voltage reward is -1; if the state of the current station terminal is determined to be "S2: single-phase ground short-circuit fault" and the first control action is "a2: open", the real-time voltage reward is 100; if the state of the current station terminal is determined to be "S2: single-phase ground short-circuit fault" and the first control action is "a3: close", the real-time voltage reward is -1.

[0117] If the state of the current station terminal is determined to be "S3: two-phase short circuit fault" and the first control action is "no action", the real-time voltage reward is -1; if the state of the current station terminal is determined to be "S3: two-phase short circuit fault" and the first control action is "a2: open", the real-time voltage reward is 100; if the state of the current station terminal is determined to be "S3: two-phase short circuit fault" and the first control action is "a3: close", the real-time voltage reward is -1.

[0118] If the state of the current station terminal is determined to be "S4: two-phase ground short-circuit fault" and the first control action is "no action", the real-time voltage reward is -1; if the state of the current station terminal is determined to be "S4: two-phase ground short-circuit fault" and the first control action is "a2: open", the real-time voltage reward is 100; if the state of the current station terminal is determined to be "S4: two-phase ground short-circuit fault" and the first control action is "a3: close", the real-time voltage reward is -1.

[0119] If the current state of the station terminal is determined to be "S5: three-phase short circuit fault" and the first control action is "no action", the real-time voltage reward is -1; if the current state of the station terminal is determined to be "S5: three-phase short circuit fault" and the first control action is "a2: open", the real-time voltage reward is 100; if the current state of the station terminal is determined to be "S5: three-phase short circuit fault" and the first control action is "a3: close", the real-time voltage reward is -1.

[0120] According to the technical solution provided by the embodiment of the present disclosure, obtaining real-time rewards through the above solution can improve the accuracy of the obtained real-time rewards, help improve the control efficiency of the station terminal, and reduce the control cost of the station terminal.

[0121] Figure 2 The structural block diagram of the station terminal control device according to the embodiment of the present disclosure is shown. The device can be implemented as part or all of the electronic device through software, hardware or a combination of both.

[0122] like Figure 2 As shown, the station terminal control device 200 includes:

[0123] The state acquisition module 201 is configured to acquire a pre-trained target station terminal control model and a current station terminal state of the station terminal, wherein the target station terminal control model includes an action value function value table, and the action value function value table is used to indicate the correspondence between the station terminal state, the control action and the action value;

[0124] The action acquisition module 202 is configured to determine a target control action corresponding to a target action value based on a target station terminal control model, wherein the target action value is a maximum action value corresponding to a current station terminal state;

[0125] The station terminal control module 203 is configured to control the station terminal to perform a target control action.

[0126] In one implementation of the present disclosure, the state acquisition module 201 is specifically configured as follows:

[0127] Obtaining an initial station terminal control model;

[0128] Acquire a first station terminal state of the current station terminal, and determine a first control action corresponding to a first action value based on a station terminal control model, wherein the first action value is a maximum action value corresponding to the first station terminal state, and the station terminal control model includes an action value function value table, and the action value function value table is used to indicate a corresponding relationship between the station terminal state, the control action, and the action value;

[0129] Acquire a second station terminal state of the station terminal after the station terminal executes the first control action and a distribution network parameter detected by the station terminal;

[0130] Obtaining a second action value based on the station terminal control model, wherein the second action value is a maximum action value corresponding to the second station terminal state;

[0131] Acquire an updated action value based on the first control action, the distribution network parameter, the first action value, and the second action value, and update the first action value in the action value function value table based on the updated action value;

[0132] In response to the action-value function value table converging, the station terminal control model is determined as the target station terminal control model.

[0133] In one implementation of the present disclosure, the state acquisition module 201 is specifically configured as follows:

[0134] Obtaining a real-time reward R(s,a) based on the first control action and the distribution network parameters;

[0135] Based on Q g(s,a)=Q1(s,a)+α[R(s,a)+γmaxQ(s′,a′)-Q1(s,a)] to get the updated action value Q g (s,a), where Q1(s,a) is the first action value, s is the terminal state of the first station, a is the first control action, α is a preset learning rate parameter, γ is a preset conversion factor, maxQ(s′,a′) is the second action value, s′ is the terminal state of the second station, and a′ is the second control action.

[0136] In an implementation of the present disclosure, the distribution network parameter includes at least one of a current value, a voltage value, current waveform data within a preset time length, and voltage waveform data within a preset time length.

[0137] In one implementation of the present disclosure, the state acquisition module 201 is specifically configured as follows:

[0138] Based on the first control action, the current value in the distribution network parameters and the current waveform data within the preset time, a current real-time reward R1(s,a) is obtained, and based on the first control action, the voltage value in the distribution network parameters and the voltage waveform data within the preset time, a voltage real-time reward R2(s,a) is obtained;

[0139] Obtain the real-time reward R(s,a) based on R(s,a)=k1R1(s,a)+k2R2(s,a).

[0140] In one implementation of the present disclosure, the control action includes at least one of no action, closing, and opening;

[0141] And / or, the station terminal status includes at least one of no fault, single-phase ground short circuit fault, two-phase short circuit fault, two-phase ground short circuit fault, and three-phase short circuit fault.

[0142] According to the technical solution provided by the embodiment of the present disclosure, by obtaining the pre-trained target station terminal control model and the current station terminal state of the station terminal; based on the target station terminal control model, determine the target control action corresponding to the target action value, wherein the target action value is the maximum action value corresponding to the current station terminal state; control the station terminal to execute the target control action. This solution enables the station terminal to execute the target control action that matches the maximum action value corresponding to the current station terminal state without the intervention of the cloud or human-computer interaction terminal, thereby ensuring that the station terminal state of the station terminal after executing the target control action can meet the requirements of normal operation. Therefore, this solution improves the control efficiency of the station terminal and reduces the control cost.

[0143] In an embodiment of the present disclosure, a station terminal is provided. Figure 3A structural block diagram of a station terminal according to an embodiment of the present disclosure is shown, Figure 3 As shown, the station terminal 300 includes the station terminal control device 200 in any of the above embodiments.

[0144] In one implementation of the present disclosure, the station terminal is further used for:

[0145] Collect terminal working data and distribution network data;

[0146] Send terminal working data and distribution network data to the distribution automation master station;

[0147] The receiving distribution automation master station determines the station terminal control model based on the collected terminal working data and distribution network data.

[0148] In an embodiment of the present disclosure, a distribution automation master station is provided, and the distribution automation master station is used for:

[0149] The terminal working data and distribution network data uploaded by the terminal at the receiving station;

[0150] The station terminal control model is determined based on the terminal working data and the distribution network data, and the station terminal control model is sent to the station terminal.

[0151] In an embodiment of the present disclosure, a power distribution network control system is provided. Figure 4 A structural block diagram of a power distribution network control system according to an embodiment of the present disclosure is shown as follows: Figure 4 As shown, the distribution network control system includes the station terminal 401 and the distribution automation master station 402 in any of the above embodiments.

[0152] The present disclosure also discloses an electronic device, Figure 5 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0153] like Figure 5 As shown, the electronic device includes a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to an embodiment of the present disclosure.

[0154] The present disclosure provides a method for controlling a terminal at a station, including:

[0155] Obtaining a pre-trained target station terminal control model and a current station terminal state of the station terminal, wherein the target station terminal control model includes an action value function value table, and the action value function value table is used to indicate the corresponding relationship between the station terminal state, the control action, and the action value;

[0156] Based on the target station terminal control model, determine the target control action corresponding to the target action value, where the target action value is the maximum action value corresponding to the current station terminal state;

[0157] The terminal at the control station executes the target control action.

[0158] In one implementation of the present disclosure, obtaining a pre-trained target station terminal control model includes:

[0159] Obtaining an initial station terminal control model;

[0160] Acquire a first station terminal state of the current station terminal, and determine a first control action corresponding to a first action value based on a station terminal control model, wherein the first action value is a maximum action value corresponding to the first station terminal state, and the station terminal control model includes an action value function value table, and the action value function value table is used to indicate a corresponding relationship between the station terminal state, the control action, and the action value;

[0161] Acquire a second station terminal state of the station terminal after the station terminal executes the first control action and a distribution network parameter detected by the station terminal;

[0162] Obtaining a second action value based on the station terminal control model, wherein the second action value is a maximum action value corresponding to the second station terminal state;

[0163] Acquire an updated action value based on the first control action, the distribution network parameter, the first action value, and the second action value, and update the first action value in the action value function value table based on the updated action value;

[0164] In response to the action-value function value table converging, the station terminal control model is determined as the target station terminal control model.

[0165] In one implementation of the present disclosure, obtaining an update action value based on the first control action, the distribution network parameter, the first action value, and the second action value includes:

[0166] Obtaining a real-time reward R(s,a) based on the first control action and the distribution network parameters;

[0167] Based on Q g (s,a)=Q1(s,a)+α[R(s,a)+γmaxQ(s′,a′)-Q1(s,a)] to get the updated action value Q g (s,a), where Q1(s,a) is the first action value, α is a preset learning rate parameter, γ is a preset reduction factor, and maxQ(s′,a′) is the second action value.

[0168] In an implementation of the present disclosure, the distribution network parameter includes at least one of a current value, a voltage value, current waveform data within a preset time length, and voltage waveform data within a preset time length.

[0169] In one implementation of the present disclosure, obtaining a real-time reward R(s,a) based on the first control action and the distribution network parameters includes:

[0170] Based on the first control action, the current value in the distribution network parameters and the current waveform data within the preset time, a current real-time reward R1(s,a) is obtained, and based on the first control action, the voltage value in the distribution network parameters and the voltage waveform data within the preset time, a voltage real-time reward R2(s,a) is obtained;

[0171] The real-time reward R(s,a) is obtained based on R(s,a)=k1R1(s,a)+k2R2(s,a), where k1 is the preset current reward weight and k2 is the preset voltage reward weight.

[0172] In one implementation of the present disclosure, the control action includes at least one of no action, closing, and opening;

[0173] And / or, the station terminal status includes at least one of no fault, single-phase ground short circuit fault, two-phase short circuit fault, two-phase ground short circuit fault, and three-phase short circuit fault.

[0174] Figure 6 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0175] like Figure 6 As shown, the computer system includes a processing unit, which can perform the various methods in the above-mentioned embodiments according to the program stored in the read-only memory (ROM) or the program loaded from the storage part into the random access memory (RAM). In the RAM, various programs and data required for the operation of the computer system are also stored. The processing unit, ROM and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0176] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs a communication process via a network such as the Internet. The drive is also connected to the I / O interface as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed. Among them, the processing unit can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0177] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the above method. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium.

[0178] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0179] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or programmable hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0180] As another aspect, the present disclosure further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores one or more programs, and the programs are used by one or more processors to execute the method described in the present disclosure.

[0181] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. A station terminal control method, characterized in that: include: Acquire a pre-trained target station terminal control model and a current station terminal state of the station terminal, wherein the target station terminal control model includes an action value function value table, and the action value function value table is used to indicate the corresponding relationship between the station terminal state, the control action, and the action value; Based on the target station terminal control model, determining a target control action corresponding to a target action value, wherein the target action value is a maximum action value corresponding to the current station terminal state; Control the station terminal to execute the target control action.

2. The station terminal control method according to claim 1, characterized in that: Before obtaining the pre-trained target station terminal control model, the method further includes: Obtaining an initial station terminal control model; Acquire a first station terminal state of the current station terminal, and determine a first control action corresponding to a first action value based on the station terminal control model, wherein the first action value is a maximum action value corresponding to the first station terminal state, and the station terminal control model includes an action value function value table, and the action value function value table is used to indicate the correspondence between the station terminal state, the control action, and the action value; Acquire a second station terminal state of the station terminal after the station terminal executes the first control action and a distribution network parameter detected by the station terminal; Acquire a second action value based on the station terminal control model, wherein the second action value is a maximum action value corresponding to the second station terminal state; Acquire an updated action value based on the first control action, the power distribution network parameter, the first action value, and the second action value, and update the first action value in the action value function value table based on the updated action value; In response to the action-value function value table converging, the station terminal control model is determined as the target station terminal control model.

3. The station terminal control method according to claim 2, characterized in that: The obtaining and updating the action value based on the first control action, the power distribution network parameter, the first action value, and the second action value includes: Obtaining a real-time reward R(s,a) based on the first control action and the distribution network parameter; Based on Q g (s,a)=Q1(s,a)+α[R(s,a)+γmaxQ(s′,a′)-Q1(s,a)] to get the updated action value Q g (s,a), where Q1(s,a) is the first action value, s is the terminal state of the first station, a is the first control action, α is a preset learning rate parameter, γ is a preset conversion factor, maxQ(s′,a′) is the second action value, s′ is the terminal state of the second station, and a′ is the second control action.

4. The station terminal control method according to claim 3, characterized in that: The distribution network parameters include at least one of a current value, a voltage value, current waveform data within a preset time length, and voltage waveform data within a preset time length.

5. The station terminal control method according to claim 4, characterized in that: The acquiring of a real-time reward R(s,a) based on the first control action and the distribution network parameter includes: Based on the first control action, the current value in the distribution network parameter and the current waveform data within a preset time, a current real-time reward R1(s,a) is obtained, and based on the first control action, the voltage value in the distribution network parameter and the voltage waveform data within a preset time, a voltage real-time reward R2(s,a) is obtained; The real-time reward R(s,a) is obtained based on R(s,a)=k1R1(s,a)+k2R2(s,a), wherein k1 is a preset current reward weight, and k2 is a preset voltage reward weight.

6. The station terminal control method according to claim 2, characterized in that: The control action includes at least one of no action, closing, and opening; And / or, the station terminal status includes at least one of no fault, single-phase ground short circuit fault, two-phase short circuit fault, two-phase ground short circuit fault, and three-phase short circuit fault.

7. A station terminal control device, characterized in that: include: A state acquisition module is configured to acquire a pre-trained target station terminal control model and a current station terminal state of the station terminal, wherein the target station terminal control model includes an action value function value table, and the action value function value table is used to indicate the corresponding relationship between the station terminal state, the control action and the action value; An action acquisition module is configured to determine a target control action corresponding to a target action value based on the target station terminal control model, wherein the target action value is a maximum action value corresponding to the current station terminal state; The station terminal control module is configured to control the station terminal to execute the target control action.

8. The station terminal control device according to claim 7, characterized in that: The state acquisition module is specifically configured as follows: Obtaining the initial station terminal control model; Acquire a first station terminal state of the current station terminal, and determine a first control action corresponding to a first action value based on the station terminal control model, wherein the first action value is a maximum action value corresponding to the first station terminal state, and the station terminal control model includes an action value function value table, and the action value function value table is used to indicate the correspondence between the station terminal state, the control action, and the action value; Acquire a second station terminal state of the station terminal after the station terminal executes the first control action and a distribution network parameter detected by the station terminal; Acquire a second action value based on the station terminal control model, wherein the second action value is a maximum action value corresponding to the second station terminal state; Acquire an updated action value based on the first control action, the power distribution network parameter, the first action value, and the second action value, and update the first action value in the action value function value table based on the updated action value; In response to the action-value function value table converging, the station terminal control model is determined as the target station terminal control model.

9. The station terminal control device according to claim 8, characterized in that: The state acquisition module is specifically configured as follows: Obtaining a real-time reward R(s,a) based on the first control action and the distribution network parameter; Based on Q g (s,a)=Q1(s,a)+α[R(s,a)+γmaxQ(s′,a′)-Q1(s,a)] to get the updated action value Q g (s,a), where Q1(s,a) is the first action value, s is the terminal state of the first station, a is the first control action, α is a preset learning rate parameter, γ is a preset conversion factor, maxQ(s′,a′) is the second action value, s′ is the terminal state of the second station, and a′ is the second control action.

10. The station terminal control device according to claim 9, characterized in that: The distribution network parameters include at least one of a current value, a voltage value, current waveform data within a preset time length, and voltage waveform data within a preset time length.

11. The station terminal control device according to claim 10, characterized in that: The state acquisition module is specifically configured as follows: Based on the first control action, the current value in the distribution network parameter and the current waveform data within a preset time, a current real-time reward R1(s,a) is obtained, and based on the first control action, the voltage value in the distribution network parameter and the voltage waveform data within a preset time, a voltage real-time reward R2(s,a) is obtained; The real-time reward R(s,a) is obtained based on R(s,a)=k1R1(s,a)+k2R2(s,a), wherein k1 is a preset current reward weight, and k2 is a preset voltage reward weight.

12. The station terminal control device according to claim 8, characterized in that: The control action includes at least one of no action, closing, and opening; And / or, the station terminal status includes at least one of no fault, single-phase ground short circuit fault, two-phase short circuit fault, two-phase ground short circuit fault, and three-phase short circuit fault.

13. A station terminal, characterized in that: The station terminal includes the station terminal control device according to any one of claims 7-12.

14. An electronic device, characterized in that: The method comprises a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps described in any one of claims 1 to 6.

Images