Ordered charging control method, device and system for power distribution network

Abstract

The invention provides an ordered charging control method, device and system for a power distribution network, and the method comprises the steps: obtaining the data of an electrical load and a charging load of a power distribution area, and building an ordered charging target model of the power distribution area with the minimum load fluctuation of the power distribution area as a target; converting the target model into a Markov decision process, and constructing a Q learning algorithm model oriented to ordered charging scheduling; and obtaining an access strategy of each charging load according to the Q learning algorithm model and the target model. According to the technical scheme provided by the invention, the influence of the access charging load on the fluctuation of the power distribution network load is reduced, the peak load shifting of the power distribution network load is realized, and the operation stability of the power distribution network is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of orderly charging, and in particular to an orderly charging control method, device and system for a distribution network. Background Art

[0002] As the number of electric vehicles connected to the grid continues to increase, the demand for charging capacity provided by the distribution network has increased, but the existing distribution network has limited carrying capacity and is unable to support the new high charging capacity. The existing distribution network is already facing the risk of overloading of regional lines and distribution transformers and reduced stability of distribution network operation.

[0003] Distributed power sources, energy storage and other resources can improve the distribution network's ability to accept charging loads such as electric vehicles to a certain extent. However, various charging loads, as important adjustable, controllable and flexible resources on the power consumption side, have become one of the largest loads in the power grid, with randomness, simultaneity and impact. The difficulty of maintaining safe and stable operation by traditional means of power grid regulation will be greatly increased, and it cannot be applied to the power regulation scenario of the new distribution system with large-scale charging load access. Summary of the invention

[0004] The present invention provides a method, device and system for orderly charging control of a distribution network, which reduce the impact of connected charging load on load fluctuation of the distribution network, realize peak shaving and valley filling of the distribution network load, and improve the stability of distribution network operation.

[0005] In a first aspect, an embodiment of the present invention provides an orderly charging control method for a distribution network, comprising:

[0006] Obtain the data of power consumption load and charging load in the distribution substation area, and establish a target model for orderly charging in the distribution substation area with the goal of minimizing load fluctuation in the distribution substation area;

[0007] Convert the target model into a Markov decision process and construct a Q-learning algorithm model for orderly charging scheduling;

[0008] The access strategy of each charging load is obtained according to the Q learning algorithm model and the target model.

[0009] Optionally, the data of power load and charging load in the distribution substation area are obtained, and a target model for orderly charging in the distribution substation area is established with the minimum load fluctuation in the distribution substation area as the goal, including:

[0010] Obtain the predicted power load at each time in the distribution station area, the number of charging loads at each time, and the power of a single charging load;

[0011] Obtaining the total load of the distribution station area at each moment according to the predicted power load at each moment, the number of the charging loads and the power of the single charging load;

[0012] The minimum load fluctuation is obtained according to the difference between the total load and the average load of the distribution station area at each moment, so as to establish the target model.

[0013] Optionally, the total load of the distribution station area at each moment is expressed as:

[0014] P i =I i ·P c +P load (i)

[0015] I i is the number of charging loads at time Ti, P c is the power of a single charging load, P load (i) is the predicted power load of the distribution station area at time Ti;

[0016] The target model is expressed as:

[0017]

[0018] Among them, P i is the total load of the distribution station area at time Ti, P avg is the average load, P MAX It is the upper limit of the load in the distribution station area.

[0019] Optionally, the target model is converted into a Markov decision process to construct a Q learning algorithm model for orderly charging scheduling, including:

[0020] According to the Markov decision process, a triplet including a state space, an action space and a reward function is set, wherein the state space represents the total load of the distribution station area at each moment, the action space represents the action of scheduling the newly added charging load to charge in the preset time period, and the reward function represents the total amount of the charging load after the action;

[0021] According to the state space and the action space, a Q value function representing the action taken by the distribution station area under the current system state is obtained.

[0022] Optionally, the preset time period is a low period of the total load of the distribution station area.

[0023] Optionally, obtaining an access strategy for each charging load according to the Q learning algorithm model and the target model includes:

[0024] Establishing an equation relationship between the Q value function and the target model;

[0025] Taking the target model as the minimum value as the target, solving the Q value function according to the current input state space, thereby obtaining the access strategy with the target model as the minimum value.

[0026] Optionally, after converting the target model into a Markov decision process and constructing a Q learning algorithm model for orderly charging scheduling, the following further includes:

[0027] Training the Q learning algorithm model;

[0028] Wherein, training the Q learning algorithm model includes:

[0029] S1. Observe the current system status during the training cycle;

[0030] S2, select actions according to the random strategy to obtain the reward function of the next system state;

[0031] S3, according to the action space of the current system state after the action, obtain the next system state;

[0032] S4, update the Q value function of the current system state according to the next system state and the reward function of the next system state;

[0033] S5. Repeat steps S2 to S4 until the training cycle ends.

[0034] Optionally, the Q-value function of the current system state is updated according to the next system state and the reward function R of the next system state, including:

[0035] Update the Q value of the current system state according to the update function, and the update function is:

[0036]

[0037] Among them, α is the learning rate, β is the discount rate; Q old (s (t) ,a (t) ) represents the Q value of taking action under the current system state; Q new (s (t) ,a (t) ) represents the updated Q value under the current system state; R (t+1) is the reward function of the next system state after executing the action; s (t+1) Indicates the next system state after executing the current action; It represents the minimum Q value for taking an action in the next system state.

[0038] In a second aspect, an embodiment of the present invention provides an orderly charging control device for a distribution network, comprising:

[0039] The first model building module is used to obtain the data of power load and charging load in the distribution station area, and to establish a target model for orderly charging in the distribution station area with the minimum load fluctuation in the distribution station area as the goal;

[0040] A second model building module is used to convert the target model into a Markov decision process to build a Q learning algorithm model for orderly charging scheduling;

[0041] A calculation module is used to obtain an access strategy for each charging load according to the Q learning algorithm model and the target model.

[0042] In a third aspect, an embodiment of the present invention provides an orderly charging control system for a distribution network, comprising an edge computing intelligent device and a communication module, wherein the edge computing intelligent device comprises a power consumption information collection unit, a power consumption load collection unit, a charging load collection unit, an orderly charging control device for a distribution network, and a database;

[0043] The communication module is used to establish a communication connection between the edge computing smart device and the external smart meter and charging pile;

[0044] The power consumption information collection unit is used to collect historical data and real-time data of the smart meters in the distribution station area and store them in the database;

[0045] The power load collection unit is used to predict the power load of the distribution station area on the day according to the historical data and real-time data of the power load of the charging pile, and obtain the predicted power load at each moment;

[0046] The charging load collection unit is used to collect the number of charging loads of the charging piles in the distribution station area at each moment and the power of a single charging load, and store them in the database;

[0047] The orderly charging control device of the distribution network is used to obtain data on power load and charging load in the distribution station area from the database, establish a target model for orderly charging in the distribution station area with the minimum load fluctuation in the distribution station area as the goal; convert the target model into a Markov decision process to construct a Q learning algorithm model for orderly charging scheduling; and obtain an access strategy for each charging load based on the Q learning algorithm model and the target model.

[0048] The technical solution provided by the embodiment of the present invention obtains the data of power load and charging load in the distribution station area, establishes a target model for orderly charging in the distribution station area with the minimum load fluctuation in the distribution station area as the target; converts the target model into a Markov decision process, constructs a Q learning algorithm model for orderly charging scheduling, considers the calculation with the minimum load fluctuation of the distribution network as the target to reduce the complexity of the target operation, uses the Q learning algorithm model to obtain the state-action mapping, guides the access strategy of the charging load according to the current state, thereby controlling the access of the charging load, so as to reduce the impact of the access charging load on the load fluctuation of the distribution network, and realizes the peak shaving and valley filling of the distribution network load to improve the stability of the distribution network operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic flow chart of an orderly charging control method for a distribution network is provided for an embodiment of the present invention;

[0050] Figure 2 A flow chart of another orderly charging control method for a distribution network is provided for an embodiment of the present invention;

[0051] Figure 3 A schematic structural diagram of an orderly charging control device for a power distribution network is provided for an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the structure of an orderly charging control system for a power distribution network is provided for an embodiment of the present invention;

[0053] Figure 5 A schematic structural diagram of an electronic device is provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] As the number of electric vehicles connected to the grid continues to increase, the demand for charging capacity provided by the distribution network increases, while the carrying capacity of the existing distribution network is limited and it is difficult to support the new high charging capacity. The existing distribution network is already facing the risk of overload of lines and distribution transformers in the region and reduced stability of distribution network operation. Distributed power sources, energy storage and other resources can improve the distribution network's ability to accept electric vehicles to a certain extent, but the charging behavior of electric vehicles is highly random, and the output of distributed photovoltaics is highly volatile and has complex dynamic characteristics. Therefore, traditional distribution transformers are not suitable for new distribution system power regulation scenarios for multiple distributed resources and large-scale electric vehicle access.

[0056] In view of this, Figure 1 A flow chart of an orderly charging control method for a distribution network is provided for an embodiment of the present invention. This embodiment is applicable to the control of orderly charging of charging loads. The method can be executed by an orderly charging control device, which can be implemented in hardware and / or software. The method specifically includes the following steps:

[0057] S110, obtaining data on power load and charging load in the distribution substation area, and establishing a target model for orderly charging in the distribution substation area with the goal of minimizing load fluctuation in the distribution substation area;

[0058] Specifically, the distribution substation supplies power to power users, wherein the power users include conventional power users and charging users consisting of charging piles. The power usage of conventional power users can be referred to as power load, and the power usage of charging piles connected to charging users can be referred to as charging load. The historical data and / or real-time data related to power load and charging load can be collected by the acquisition unit provided in the distribution substation. Taking into account the volatility of various distributed resources and the load volatility generated by the access of electric vehicles to charging piles, a target model for the minimum load fluctuation of the distribution substation is established. With the minimum load fluctuation as a limit, the access of the charging load is controlled to reduce the variation range between the load values ​​of the distribution network and improve the transmission efficiency of the power grid. Exemplarily, the minimum load fluctuation can reflect the variation range of the distribution network load by the difference between the total load and the average load of the distribution network at each moment. Furthermore, the standard deviation of the total load and the average load of the distribution network at each moment can be used to reflect the variation range of the distribution network load. The value of the standard deviation is closer to the variation range of the original data, and the size of the standard deviation can directly reflect the size of the discreteness of the data, which helps to more intuitively evaluate the volatility of the data.

[0059] S120, converting the target model into a Markov decision process and constructing a Q learning algorithm model for orderly charging scheduling;

[0060] Specifically, the Markov Decision Process (MDP) is a mathematical model that describes the decision-making process of decision makers in an uncertain environment. The current state of the MDP depends only on the current state and action, and is independent of the historical state and action. Therefore, the target model can be converted into an MDP problem to construct a Q learning algorithm model. The Q learning algorithm model solves the MDP problem by learning an action Q value function to evaluate the expected utility of taking an action in a given state. Exemplarily, according to the MDP setting, a triple including a state space, an action space, and a reward function can be expressed as {S, A, R}, where S is the state space, A is the action space, and R is the reward function.

[0061] State space S: represents the total load of the distribution area at each moment. The system state at time slot t is set as:

[0062]

[0063] Action space A: represents the actions that affect system performance, scheduling the newly added charging load to the preset time period [T j , T j+L] to perform charging action, and set the action of time slot t as:

[0064]

[0065] That is to say, the newly added charging load is scheduled to the preset time period [T j , T j+L ] for charging, the number of charging loads at the corresponding moment of the period increases by one, and the number of charging loads at other moments remains unchanged. Therefore, if the preset time period is set as the load valley period of the distribution network, the load valley period can be fully utilized for charging in the action result, which can improve the distribution utilization rate of the distribution network.

[0066] Reward function R: Set the reward to select an action in the current system state S, get the reward function for the next time slot t+1, and enter the next system state S (t+1) The reward function R can be expressed as the sum of the number of charging loads after an action:

[0067]

[0068] Taking action space A in state space S results in the Q-value function, i.e., the Q-learning algorithm model, denoted as Q(s,a).

[0069] S130 . Obtaining access strategies for various charging loads according to the Q learning algorithm model and the target model.

[0070] Specifically, a relationship is established between the Q-value function and the target model, and the target of the Q-value function is set to the minimum value of the target model for solving to obtain the Q value. That is, one system state and one action correspond to a Q value. The Q value maps each state-action pair to a real number, which represents the minimum load fluctuation expected after executing the action from the state. In the Q-Learning algorithm, the Q value is updated accordingly by continuously interacting with the environment, observing the rewards obtained by taking different actions in different states, and gradually obtaining the optimal value of the target model. Therefore, according to the Q value that meets the minimum load fluctuation, that is, the target model is the minimum value, the corresponding state-action mapping is determined, and the charging load access strategy can be guided according to the current state.

[0071] The technical solution provided by the embodiment of the present invention obtains the data of power load and charging load in the distribution station area, establishes a target model for orderly charging in the distribution station area with the minimum load fluctuation in the distribution station area as the target; converts the target model into a Markov decision process, constructs a Q learning algorithm model for orderly charging scheduling, considers the calculation with the minimum load fluctuation of the distribution network as the target to reduce the complexity of the target operation, uses the Q learning algorithm model to obtain the state-action mapping, guides the access strategy of the charging load according to the current state, thereby controlling the access of the charging load, so as to reduce the impact of the access charging load on the load fluctuation of the distribution network, and realizes the peak shaving and valley filling of the distribution network load to improve the stability of the distribution network operation.

[0072] Figure 2 A flow chart of another orderly charging control method for a distribution network is provided for an embodiment of the present invention, see Figure 2 ,include:

[0073] S210, obtaining the predicted power load at each time in the distribution station area, the number of charging loads at each time, and the power of a single charging load;

[0074] Specifically, the power load of the distribution station area includes the power load of conventional users, and the power information collection unit obtains relevant data by reading the electric meter in the distribution station area. The distribution station area is usually equipped with a power load prediction function. Through the real-time data and historical data of the power load in the distribution station area, the power load of the day is predicted by using the time series method, the neural network prediction method, etc., so as to obtain the predicted power load at each moment of the distribution station area. Among them, it should be noted that the above-mentioned power load prediction method is a related prior art and will not be repeated here. The above-mentioned moment can be a time point, such as 24 hours a day, and the time point corresponding to each hour is regarded as a moment. The moment can also be a time period, such as 24 hours a day, and each hour time period is regarded as a moment. There is no specific limitation here, and the corresponding moment definition can be set according to the application requirements. In the embodiment of the present invention, taking the time period corresponding to the moment as an example, the charging load is mainly the new energy vehicle connected to the charging pile, and the number of charging loads at each moment can be the number of charging piles connected in a time period. The power of a single charging load can be the power demand of the charging pile.

[0075] S220, obtaining the total load of the distribution station area at each moment according to the predicted power load at each moment, the number of charging loads and the power of a single charging load;

[0076] Specifically, the total load of the distribution area at each moment includes the user's power load and the connected charging load. Therefore, the total load of the distribution area at each moment can be expressed as the sum of the predicted power load at each moment and the connected charging load. Among them, the connected charging load can be expressed as the product of the number of charging loads and the power of a single charging load.

[0077] Therefore, the total load of the distribution station area at each moment is expressed as:

[0078] P i =I i ·P c +P load (i) (4)

[0079] Among them, I i is the number of charging loads at time Ti, P c is the power of a single charging load, P load (i) is the predicted power load of the distribution station area at time Ti;

[0080] S230. Establish a target model according to the minimum load fluctuation between the total load and the average load of the distribution station area at each moment.

[0081] Specifically, the minimum load fluctuation can reflect the change range of the distribution network load through the difference between the total load and the average load of the distribution network at each moment. Furthermore, the standard deviation of the total load and the average load of the distribution network at each moment can be used. The value of the standard deviation is closer to the change range of the original data. The size of the standard deviation can directly reflect the size of the data discreteness, which helps to more intuitively evaluate the volatility of the data.

[0082] Therefore, the target model is expressed as:

[0083]

[0084] Among them, P i is the total load of the distribution station area at time Ti, P avg is the average load, P MAX The upper limit of the load in the distribution area. The total load in the distribution area is less than or equal to the upper limit of the load in the distribution area as a constraint of the target model to avoid the total load in the distribution area exceeding the upper limit of the load in the distribution area, resulting in insufficient power of the distribution network or reduced stability of the distribution network.

[0085] S240. According to the MDP, a triplet including a state space S, an action space A and a reward function R is set, wherein the state space S represents the total load of the distribution station area at each moment, the action space A represents the action of scheduling the newly added charging load to the preset time period, and the reward function R represents the total number of charging loads after the action; according to the state space S and the action space A, a Q-value function representing the action taken by the distribution station area under the current system state is obtained.

[0086] Specifically, the target model is converted into an MDP problem and a Q-learning algorithm model is constructed. The Q-learning algorithm model evaluates the expected utility of taking an action in a given state by learning an action Q-value function, thereby solving the MDP problem. According to the MDP setting, a triple consisting of state space, action space, and reward function can be expressed as {S, A, R}, where S is the state space, A is the action space, and R is the reward function.

[0087] The state space S represents the total load of the distribution area at each time, and the system state of the time slot t is set as:

[0088]

[0089] Action space A represents the actions that affect system performance, scheduling the newly added charging load to the preset time period [T j , T j+L ] to perform charging action, and set the action of time slot t as:

[0090]

[0091] That is to say, the newly added charging load is scheduled to the preset time period [T j , T j+L ] for charging, the number of charging loads at the corresponding time of the period increases by one, and the number of charging loads at other times remains unchanged. Therefore, if the preset time period is set as the load valley period of the distribution network, the load valley period can be fully utilized for charging, thereby improving the distribution utilization rate of the distribution network.

[0092] The reward function R sets the reward as selecting an action in the current system state S, obtains the reward function for the next time slot t+1, and enters the next system state S (t+1) The reward function R can be expressed as the sum of the number of charging loads after an action:

[0093]

[0094] Taking action space A under state space S obtains the Q value function, that is, the Q learning algorithm model, denoted as Q(s,a).

[0095] S250, establish an equation relationship between the Q value function and the target model; take the target model as the minimum value, solve the Q value function according to the current input state space S to obtain an access strategy with the target model as the minimum value.

[0096] Specifically, take the action space A under the state space S to obtain the Q value function, let Q(s,a) = fp, establish a Q table, and set the Q value in the Q table to be initialized to P load(i)After executing an action in each time slot system state, the next system state and the reward obtained will be observed. During the operation, the Q value corresponding to each pair of system state and action is updated. Repeat the above process to continuously optimize the Q value function until the performance standard is met. At the same time, the updated Q table also facilitates better selection of corresponding actions in the future.

[0097] By setting the goal of MDP as a strategy to minimize fp, a series of Q values ​​corresponding to the minimum value of Q(s,a) are obtained. In other words, one system state and one action correspond to one Q value, and the Q value maps each state-action pair to a real number, which represents the minimum load fluctuation expected after executing the action from the state. Therefore, according to the Q value that meets the minimum load fluctuation, that is, the target model is the minimum value, the corresponding state-action mapping is determined, and the charging load access strategy can be guided according to the current state.

[0098] Among them, the update of Q value can be updated according to the update function, and the update function is:

[0099]

[0100] α is the learning rate, β is the discount rate; Q old (s (t) ,a (t) ) represents the Q value of taking action under the current system state; Q new (s (t) ,a (t) ) represents the updated Q value under the current system state; R (t+1) is the reward function of the next system state after executing the action; s (t+1) Indicates the next system state after executing the current action; It represents the minimum Q value for taking an action in the next system state.

[0101] Optionally, after converting the target model into a Markov decision process and constructing a Q-learning algorithm model for orderly charging scheduling, it also includes:

[0102] Train the Q learning algorithm model;

[0103] Among them, the Q learning algorithm model is trained, including:

[0104] S1. Observe the current system status during the training cycle;

[0105] Specifically, the learning rate α and the discount rate β are initialized, and the initial value of the Q table is 0, that is, the initial value of all state-actions is set to 0. At each time slot t within the training period Ts, the current system state s(t) is observed.

[0106] S2, select actions according to the random strategy to obtain the reward function R of the next system state;

[0107] Specifically, the system needs to randomly select an action based on the current state. The random action can be achieved by querying the Q table, or when using deep learning, the action is randomly generated by the neural network. Thus, the reward function R of the next system state can be obtained. (t+1) .

[0108] S3, according to the action space A after the action, obtain the next system state;

[0109] Specifically, after the action, it means that the charging load is connected, the action space A after the action is updated, and the number of charging loads at each moment is obtained. Combining equation (4) and equation (1), the next system state s can be obtained (t+1) .

[0110] S4, update the Q value function of the current system state according to the next system state and the reward function R of the next system state;

[0111] Specifically, the next system state s (t+1) and the reward function R for the next system state (t+1) Substituting into the update function, we can get the updated Q value function of the current system state

[0112] S5. Repeat steps S2 to S4 until the training cycle ends.

[0113] Figure 3 A schematic diagram of a structure of an orderly charging control device for a distribution network is provided in accordance with an embodiment of the present invention. Figure 3 ,include:

[0114] The first model building module 110 is used to obtain the data of power load and charging load in the distribution station area, and to establish a target model of orderly charging in the distribution station area with the minimum load fluctuation in the distribution station area as the goal;

[0115] A second model building module 120 is used to convert the target model into a Markov decision process and build a Q learning algorithm model for orderly charging scheduling;

[0116] The calculation module 130 is used to obtain the access strategy of each charging load according to the Q learning algorithm model and the target model.

[0117] Specifically, the distribution station area supplies power to power users, wherein the power users include regular users and users consisting of charging piles. The power usage of regular users can be called power load, and the power usage of charging piles can be called charging load. The historical data or real-time data related to the power load and the charging load can be collected by the corresponding acquisition unit of the distribution station area. Taking into account the volatility of various distributed resources and the load volatility generated by the access of electric vehicles to charging piles, the first model building module 110 uses the minimum load fluctuation of the distribution station area to establish a target model, and reduces the variation range between the load values ​​of the distribution network by limiting the minimum load fluctuation, thereby improving the transmission efficiency of the power grid. Exemplarily, the minimum load fluctuation can reflect the variation range of the distribution network load by the difference between the total load and the average load of the distribution network at each moment. Furthermore, the standard deviation of the total load and the average load of the distribution network at each moment can be used. The value of the standard deviation is closer to the variation range of the original data. The size of the standard deviation can directly reflect the size of the discrete degree of the data, which helps to more intuitively evaluate the volatility of the data.

[0118] The second model building module 120 converts the target model into an MDP problem and constructs a Q learning algorithm model. The Q learning algorithm model evaluates the expected utility of taking an action in a given state by learning an action Q value function, thereby solving the MDP problem. Exemplarily, according to the MDP setting, a triple including a state space, an action space, and a reward function can be expressed as {S, A, R}, where S is the state space, A is the action space, and R is the reward function.

[0119] The state space S represents the total load of the distribution area at each time, and the system state of the time slot t is set as:

[0120]

[0121] Action space A represents the actions that affect system performance, scheduling the newly added charging load to the preset time period [T j , T j+L ] to perform charging action, and set the action of time slot t as:

[0122]

[0123] That is to say, the newly added charging load is scheduled to the preset time period [T j , T j+L ] for charging, the number of charging loads at the corresponding moment of the period increases by one, and the number of charging loads at other moments remains unchanged. Therefore, if the preset time period is set as the load valley period of the distribution network, the load valley period can be fully utilized for charging in the action result, thereby improving the distribution utilization rate of the distribution network.

[0124] The reward function R sets the reward as selecting an action in the current system state S, obtains the reward function for the next time slot t+1, and enters the next system state S (t+1) The reward function R can be expressed as the sum of the number of charging loads after an action:

[0125]

[0126] Taking action space A in state space S obtains the Q value function, denoted as Q(s,a).

[0127] The calculation module 130 establishes a relationship between the Q value function and the target model. The target of the Q value function can be set to the minimum value of the target model for solving to obtain the Q value. The Q value maps each state-action pair to a real number, which represents the minimum load fluctuation expected after executing the action from the state. In the Q-Learning algorithm, the Q value gradually obtains the optimal value of the target model by continuously interacting with the environment, observing the rewards obtained by taking different actions in different states, and updating the Q value accordingly. Therefore, according to the Q value that meets the target model, the state-action mapping can be obtained, thereby guiding how to select the access strategy of the charging load according to the current state.

[0128] Figure 4 A schematic diagram of a control system for orderly charging of a power distribution network is provided in accordance with an embodiment of the present invention. Figure 4 , edge computing intelligent device and communication module 420, wherein the edge computing intelligent device includes a power consumption information collection unit 411, a power load collection unit 412, a charging load collection unit 413, an orderly charging control device 414 of the distribution network and a database 415;

[0129] Specifically, the edge computing smart device establishes a communication connection with the external smart meter 430 and the charging pile 440 through the communication module 420. The relevant data processed by the edge computing smart device can be output to the server or external control device through the communication interface 416, and remote data monitoring and device control can be realized. Among them, the communication module 420 can provide high-speed power line carrier communication (High-speed Power Line Communication, HPLC) or wireless communication to communicate with the smart meter 430 and the charging pile 440. The power consumption information collection unit 411 can collect the historical data and real-time data of the smart meter in the distribution station area through the communication module 420, and store them in the database 415; the power load collection unit 412 predicts the power load of the distribution station area on the same day according to the historical data and real-time data of the power load of the charging pile 440, and obtains the predicted power load at each time; illustratively, according to the real-time data and historical data of the power load in the distribution station area, the power load of the same day is predicted using the time series method, the neural network prediction method, etc., so as to obtain the predicted power load at each time of the distribution station area. It should be noted that the above-mentioned power load prediction method can be a related existing technology, which will not be described in detail here. The charging load collection unit 413 is used to collect the number of charging loads and the power of a single charging load at each moment of the charging pile 440 in the distribution station area, and store them in the database 415;

[0130] The orderly charging control device 414 of the distribution network can obtain the data of the power load and the charging load of the distribution station area from the database 415, and establish a target model for orderly charging of the distribution station area with the minimum load fluctuation of the distribution station area as the goal; the target model is converted into a Markov decision process, and a Q learning algorithm model for orderly charging scheduling is constructed, and the calculation is considered to be based on the minimum load fluctuation of the distribution network to reduce the complexity of the target operation, and the state-action mapping is obtained by using the Q learning algorithm model, and the access strategy of the charging load is guided according to the current state, so as to control the access of the charging load, so as to reduce the impact of the access charging load on the load fluctuation of the distribution network, and realize the peak shaving and valley filling of the distribution network load to improve the stability of the distribution network operation.

[0131] Figure 5A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0132] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0133] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0134] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as orderly charging control of the distribution network.

[0135] In some embodiments, the orderly charging control of the distribution network may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the orderly charging control of the distribution network described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the orderly charging control of the distribution network in any other suitable manner (e.g., by means of firmware).

[0136] Various implementations of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0138] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0139] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0140] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0141] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0142] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for orderly charging control of a distribution network, characterized in that: include: Obtain the data of power consumption load and charging load in the distribution substation area, and establish a target model for orderly charging in the distribution substation area with the goal of minimizing load fluctuation in the distribution substation area; The target model is converted into a Markov decision process to construct a Q-learning algorithm model for orderly charging scheduling; The access strategy of each charging load is obtained according to the Q learning algorithm model and the target model.

2. The orderly charging control method of the distribution network according to claim 1, characterized in that: Obtain the data of power consumption load and charging load in the distribution substation area, take the minimum load fluctuation in the distribution substation area as the goal, and establish a target model for orderly charging in the distribution substation area, including: Obtain the predicted power load at each time in the distribution station area, the number of charging loads at each time, and the power of a single charging load; Obtaining the total load of the distribution station area at each moment according to the predicted power load at each moment, the number of the charging loads at each moment, and the power of the single charging load; The minimum load fluctuation is obtained according to the difference between the total load and the average load of the distribution station area at each moment, so as to establish the target model.

3. The orderly charging control method of the distribution network according to claim 2, characterized in that: The total load of the distribution station area at each time is expressed as: P i =I i ·P c +P load (i) I i is the number of charging loads at time Ti, P c is the power of a single charging load, P load (i) is the predicted power load of the distribution station area at time Ti; The target model is expressed as: Among them, P i is the total load of the distribution area at time Ti, P avg is the average load, P MAX It is the upper limit of the load in the distribution station area.

4. The orderly charging control method of the distribution network according to any one of claims 2 to 3, characterized in that: The target model is converted into a Markov decision process, and a Q-learning algorithm model for orderly charging scheduling is constructed, including: According to the Markov decision process, a triplet including a state space, an action space and a reward function is set, wherein the state space represents the total load of the distribution station area at each moment, the action space represents the action of scheduling the newly added charging load to charge in the preset time period, and the reward function represents the total amount of the charging load after the action; According to the state space and the action space, a Q value function representing the action taken by the distribution station area under the current system state is obtained.

5. The orderly charging control method of the distribution network according to claim 4, characterized in that: The preset time period is the off-peak period of the total load of the distribution station area.

6. The orderly charging control method of the distribution network according to claim 5, characterized in that: Obtaining access strategies for each charging load according to the Q learning algorithm model and the target model includes: Establishing an equation relationship between the Q value function and the target model; Taking the target model as the minimum value as the target, solving the Q value function according to the current input state space, thereby obtaining the access strategy with the target model as the minimum value.

7. The orderly charging control method and system of the distribution network according to claim 6, characterized in that: After converting the target model into a Markov decision process and constructing a Q-learning algorithm model for orderly charging scheduling, the following steps are also included: Training the Q learning algorithm model; Wherein, training the Q learning algorithm model includes: S1. Observe the current system status during the training cycle; S2, select actions according to the random strategy to obtain the reward function of the next system state; S3, according to the action space of the current system state after the action, obtain the next system state; S4, update the Q value function of the current system state according to the next system state and the reward function of the next system state; S5. Repeat steps S2 to S4 until the training cycle ends.

8. The orderly charging control method and system of the distribution network according to claim 7, characterized in that: Update the Q value function of the current system state according to the next system state and the reward function R of the next system state, including: Update the Q value of the current system state according to the update function, and the update function is: Among them, α is the learning rate, β is the discount rate; Q old (s (t) ,a (t) ) represents the Q value of taking action under the current system state; Q new (s (t) ,a (t) ) represents the updated Q value under the current system state; R (t+1) is the reward function of the next system state after executing the action; s (t+1) Indicates the next system state after executing the current action; It represents the minimum Q value for taking an action in the next system state.

9. An orderly charging control device for a distribution network, characterized in that: include: The first model building module is used to obtain the data of power load and charging load in the distribution station area, and to establish a target model for orderly charging in the distribution station area with the minimum load fluctuation in the distribution station area as the goal; A second model building module is used to convert the target model into a Markov decision process to build a Q learning algorithm model for orderly charging scheduling; A calculation module is used to obtain an access strategy for each charging load according to the Q learning algorithm model and the target model.

10. An orderly charging control system for a distribution network, characterized in that: It includes an edge computing intelligent device and a communication module, wherein the edge computing intelligent device includes a power consumption information collection unit, a power load collection unit, a charging load collection unit, an orderly charging control device of the distribution network, and a database; The communication module is used to establish a communication connection between the edge computing smart device and the external smart meter and charging pile; The power consumption information collection unit is used to collect historical data and real-time data of the smart meters in the distribution station area and store them in the database; The power load collection unit is used to predict the power load of the distribution station area on the day according to the historical data and real-time data of the power load of the charging pile, and obtain the predicted power load at each moment; The charging load collection unit is used to collect the number of charging loads of the charging piles in the distribution station area at each moment and the power of a single charging load, and store them in the database; The orderly charging control device of the distribution network is used to obtain data on power load and charging load in the distribution station area from the database, establish a target model for orderly charging in the distribution station area with the minimum load fluctuation in the distribution station area as the goal; convert the target model into a Markov decision process to construct a Q learning algorithm model for orderly charging scheduling; and obtain an access strategy for each charging load based on the Q learning algorithm model and the target model.

Images