Interaction method of micro-grid and battery swap station

Abstract

The invention relates to the technical field of electric power, and provides an interaction method of a microgrid and a battery swap station, the interaction method is applied to the microgrid, and the interaction method comprises the steps that a first interaction parameter sent by the battery swap station is acquired, and the first interaction parameter comprises the number of rechargeable batteries and / or the number of discharged batteries of the battery swap station at the last moment; determining first local observation data of the micro-grid at the current moment based on the first interaction parameter; and based on the first local observation data, determining first scheduling action data of the micro-grid at the current moment by using a strategy network, so as to improve the energy scheduling efficiency and economy in the interaction process of the micro-grid and the battery swap station, and realize collaborative optimization operation between the micro-grid and the battery swap station.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric power, in particular to an interaction method of micro-grid and battery swap station. BACKGROUND

[0002] The increasing integration of distributed renewable energy and flexible load poses challenges to the stability and economy of the power system. The synergy between micro-grid and battery swap station helps to effectively utilize distributed renewable energy at different time and space scales, thereby improving the operation stability and economic benefits.

[0003] However, how to achieve the best interaction between the battery swap station and the micro-grid has always been a key problem to be solved in the industry. SUMMARY

[0004] The embodiment of the present application provides an interaction method of micro-grid and battery swap station, which can improve the energy scheduling efficiency and economy in the interaction process of micro-grid and battery swap station, and realizes the collaborative optimization operation between micro-grid and battery swap station.

[0005] In a first aspect, the embodiment of the present application provides an interaction method of micro-grid and battery swap station, applied to a micro-grid, the micro-grid being connected with a battery swap station, and the interaction method comprising: acquiring first interaction parameters sent by the battery swap station, the first interaction parameters comprising a number of charging batteries and / or a number of discharging batteries of the battery swap station at a previous time; determining first local observation data of the micro-grid at a current time based on the first interaction parameters; determining first scheduling action data of the micro-grid at the current time by using a strategy network based on the first local observation data, the first scheduling action data comprising second interaction parameters, the second interaction parameters comprising an electricity exchange price between the micro-grid and the battery swap station at the current time.

[0006] In combination with the first aspect, in some implementation manners, after determining the first scheduling action data of the micro-grid at the current time by using the strategy network based on the first local observation data, the interaction method further comprises: sending the second interaction parameters to the battery swap station; receiving updated first interaction parameters determined by the battery swap station based on the second interaction parameters, the updated first interaction parameters comprising the number of charging batteries and / or the number of discharging batteries of the battery swap station at the current time; updating the first scheduling action data of the micro-grid at a next time by using the strategy network based on the updated first interaction parameters, the updated first scheduling action data comprising updated second interaction parameters, the updated second interaction parameters comprising the electricity exchange price between the micro-grid and the battery swap station at the next time.

[0007] In combination with the first aspect, in some implementations, after determining the first scheduling action data of the micro-grid at the current time based on the first local observation data by using the policy network, the interaction method further includes: determining a first reward value of the micro-grid at the current time based on the first local observation data of the micro-grid at the current time and the first scheduling action data at the current time; and storing the first local observation data at the current time, the first scheduling action data, and the first reward value in the first database.

[0008] In combination with the first aspect, in some implementations, the first database stores a plurality of sets of first training samples, each set of first training samples including first historical local observation data, first historical scheduling action data, and corresponding first historical reward value at any historical time, and the policy network is obtained by the following method: determining a loss value of the policy network based on the first training samples; and updating network parameters of the policy network based on the loss value of the policy network.

[0009] In combination with the first aspect, in some implementations, the micro-grid is connected to a micro-grid cluster control system, the micro-grid cluster control system is connected to a main grid, and the method further includes: obtaining third interaction parameters sent by the micro-grid cluster control system, the third interaction parameters including a power purchase price and a power sale price of the micro-grid cluster control system at the current time; and determining the first local observation data of the micro-grid at the current time based on the first interaction parameters, including: determining the first local observation data of the micro-grid at the current time based on the first interaction parameters and the third interaction parameters.

[0010] In combination with the first aspect, in some implementations, the micro-grid is connected to a micro-grid cluster control system, the micro-grid cluster control system is connected to a main grid, and the method further includes: obtaining third interaction parameters sent by the micro-grid cluster control system, the third interaction parameters including a power purchase price and a power sale price of the micro-grid cluster control system at the current time; and determining the first local observation data of the micro-grid at the current time based on the first interaction parameters, including: determining the first local observation data of the micro-grid at the current time based on the first interaction parameters and the third interaction parameters.

[0011] In some implementations, after the second scheduling action data of the battery swap station at the current time is generated according to the Q values of the scheduling actions of the battery swap station, the interaction method further includes: sending the updated first interaction parameter to the micro-grid; receiving updated second interaction parameters determined by the micro-grid based on the updated first interaction parameter, the updated second interaction parameters including an electricity exchange price between the micro-grid and the battery swap station at the next time; and updating the second scheduling action data of the battery swap station at the next time by using the main network based on the updated second interaction parameters, the second scheduling action data of the battery swap station at the next time including the number of charged batteries and / or the number of discharged batteries of the battery swap station at the next time.

[0012] In some implementations, after the second scheduling action data of the battery swap station at the current time is generated according to the Q values of the scheduling actions of the battery swap station, the interaction method further includes: determining a second reward value of the battery swap station at the current time based on the second local observation data of the battery swap station at the current time and the second scheduling action data at the current time; and storing the second local observation data at the current time, the second scheduling action data and the second reward value in the second database.

[0013] In some implementations, the second database stores a plurality of second training samples, each second training sample including second historical local observation data, second historical scheduling action data and a corresponding second historical reward value at any historical time, and the main network is obtained by the following method: determining a first Q value of the second training sample by using a target network based on the second training sample, wherein the network parameters of the target network are periodically copied from the network parameters of the main network; determining a second Q value of the second training sample by using the main network based on the second training sample; determining a loss value of the main network based on the first Q value and the second Q value; and updating the network parameters of the main network based on the loss value of the main network.

[0014] In some implementations, the second scheduling action data of the battery swap station at the current time is generated according to the Q values of the scheduling actions of the battery swap station, including: generating a random number; if the random number is less than a probability threshold, randomly selecting an action from the scheduling actions to generate the second scheduling action data of the battery swap station at the current time; and if the random number is greater than or equal to the probability threshold, selecting an action with the largest Q value from the scheduling actions of the battery swap station to generate the second scheduling action data of the battery swap station at the current time.

[0015] The technical scheme of the embodiments of the present application realizes dynamic collaborative scheduling between the micro-grid and the battery swap station based on the interaction parameters and the joint strategy network and the main network, and can quickly adjust the energy interaction strategy of the two parties according to real-time key parameters such as the electricity exchange price and the number of battery charging and discharging. Such an energy interaction strategy can optimize the battery scheduling efficiency of the battery swap station and reduce the overall operating cost. Attached Figure Description

[0016] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 The above is a schematic diagram of the structure of an energy interaction system provided in an embodiment of this application.

[0018] Figure 2 The diagram shown is a flowchart illustrating an interaction method between a microgrid and a battery swapping station provided in an embodiment of this application.

[0019] Figure 3 The diagram shown is a flowchart illustrating another interaction method between a microgrid and a battery swapping station provided in an embodiment of this application.

[0020] Figure 4 The diagram shown is a schematic of an algorithm architecture provided in an embodiment of this application.

[0021] Figure 5 The diagram shown is a schematic diagram of information interaction between a microgrid and a battery swapping station provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Before introducing the interaction method between microgrids and battery swapping stations provided in the embodiments of this application, the relevant concepts and application overview involved in the embodiments of this application are introduced as follows.

[0024] A microgrid cluster control system is a system that centrally coordinates and manages multiple microgrids within a microgrid cluster. Its main functions include achieving power balance regulation between each microgrid and the main grid, optimizing the overall allocation efficiency of energy resources within the region, ensuring the safe and stable operation of the microgrid cluster, and improving the absorption capacity of distributed energy.

[0025] A microgrid refers to a small power generation and distribution system formed by distributed power sources, energy storage power stations, energy conversion devices, related loads and monitoring and protection devices, and can realize self-control, protection and management, and can operate in parallel with an external main grid or independently under certain conditions. Illustratively, a microgrid can include a wind turbine, a photovoltaic device, a diesel generator set, an energy storage power station and a load. The wind turbine is used for wind power generation, the photovoltaic device is used for solar power generation, the diesel generator set is used for diesel power generation, the energy storage power station is used for storage and management of electric energy, and the load is used for consumption of electric energy. The wind turbine and the photovoltaic device belong to distributed renewable energy sources.

[0026] A battery swap station is a place for providing battery replacement services for electric vehicles, and can meet the rapid energy supplement demand of electric vehicle users by centralized storage, charging and replacement of batteries. Illustratively, a battery swap station can include a battery swap station controller, a battery charging and discharging work station, an electric vehicle battery replacement work station and a battery storage warehouse.

[0027] The synergy between the microgrid and the battery swap station helps to effectively utilize distributed renewable energy sources at different time and space scales, thereby improving operational stability and economic benefits. However, the inventors have found in the process of implementing the present application that realizing the optimal interaction between the battery swap station and the microgrid involves the following key issues. 1) The time and space uncertainty of distributed power sources, combined with the random and unpredictable behavior of numerous electric vehicles, puts a huge burden on calculation and communication; 2) Because of the time and space uncertainty of the above-mentioned distributed power sources, the random and unpredictable behavior of numerous electric vehicles, the microgrid and the battery swap station cannot obtain complete information, and therefore it is difficult to develop an accurate coordination control strategy for the battery swap station and the microgrid; 3) As different stakeholders, the microgrid and the battery swap station must optimize their respective decision-making behaviors to achieve a balance of interests. Therefore, to obtain higher operating benefits, it is crucial to establish a coordination control architecture and a coordination control strategy for the microgrid and the battery swap station.

[0028] To solve the above problems, the embodiments of the present application propose an interaction method of a microgrid and a battery swap station to generate scheduling action data of the microgrid and the battery swap station, and realize coordinated control of the microgrid and the battery swap station.

[0029] The application scenario of the embodiments of the present application can be any energy interaction system including a microgrid and a battery swap station. The microgrid in the energy interaction system can be one or more, and each microgrid can be connected to one or more battery swap stations. Optionally, when the energy interaction system includes multiple microgrids, the energy interaction system can further include a microgrid cluster control system. In some implementation modes, see Figure 1The energy interaction system 10 can be structured as follows: a microgrid cluster control system 11 connects to multiple energy interaction branches 12. Each energy interaction branch 12 includes a microgrid 121 and a battery swapping station 122 connected to the microgrid 121. The microgrid 121 can exchange electrical energy with other microgrids 121 through the interactive microgrid cluster control system 11. The microgrid cluster control system 11 collects real-time electricity consumption data from the microgrid 121 via a communication network to ensure supply and demand balance. The battery swapping station 122 needs to meet the battery swapping needs of electric vehicles while interacting with the microgrid 121. The battery swapping station 122 provides battery swapping services to arriving electric vehicles by replacing depleted batteries with fully charged ones. The battery swapping station 122 sells surplus energy to the microgrid 121 during periods of high electricity prices and charges depleted batteries during periods of low electricity prices. Optionally, the energy interaction system can also have other architectures, which are not limited in this application embodiment. The method provided in this application embodiment can be adapted to different application scenarios by adjusting parameters, and such adjustments are all within the protection scope of this application.

[0030] Figure 2 The diagram shown is a flowchart illustrating an interaction method between a microgrid and a battery swapping station according to an embodiment of this application. See also... Figure 2 This application provides a method for interaction between a microgrid and a battery swapping station. The method is applied to a microgrid and the battery swapping station are connected. The method includes the following steps.

[0031] S210, obtain the first interaction parameters sent by the battery swapping station.

[0032] The first interaction parameter is the interaction parameter between the battery swapping station and the microgrid, reflecting the battery resource status and energy interaction demand of the battery swapping station. For example, the first interaction parameter may include the number of charging and / or discharging batteries at the battery swapping station at the previous moment. Optionally, the first interaction parameter may also include other data exchanged between the battery swapping station and the microgrid.

[0033] In some implementations, the microgrid is connected to a microgrid cluster control system, which in turn is connected to the main power grid. The interaction method between the microgrid and the battery swapping station provided in this application further includes: obtaining third interaction parameters sent by the microgrid cluster control system. These third interaction parameters are the interaction parameters between the microgrid cluster control system and the microgrid and / or the main power grid, reflecting the overall operating status of the main power grid and its scheduling requirements for the microgrid cluster. For example, the third interaction parameters include the electricity purchase price and electricity sales price of the microgrid cluster control system at the current moment.

[0034] Optionally, the purchase price and sales price of electricity at the current moment can be set as needed by the microgrid cluster control system.

[0035] In some implementations, the method for determining the purchase price and sales price of electricity for the microgrid cluster control system at the current moment can be: determining the remaining power of the microgrid cluster control system; and determining the purchase price and sales price of electricity for the microgrid cluster control system based on the remaining power, net output power, net demand power of the microgrid cluster control system, and the electricity exchange price between the microgrid and the battery swapping station.

[0036] In some implementations, the purchase price and sale price of the microgrid cluster control system are determined based on the remaining power, net output power, net demand power of the microgrid cluster control system, and the power exchange price between the microgrid and the battery swapping station. This includes: if the remaining energy of the microgrid cluster control system is greater than or equal to a first threshold, then the sale price of the microgrid cluster control system is determined based on the power exchange price between the microgrid and the battery swapping station; if the remaining energy of the source management community is less than the first threshold, then the sale price of the microgrid cluster control system is determined based on the ratio of the sum of a first product and the remaining energy of the microgrid cluster control system to the net demand power of the microgrid cluster control system. The first product is determined by multiplying the power exchange price of the microgrid by the net demand power of the microgrid cluster control system.

[0037] In some implementations, the purchase price and sale price of electricity for the microgrid cluster control system are determined based on the remaining power, net output power, net demand power, and the electricity exchange price between the microgrid and the battery swapping station. This includes: if the remaining energy of the microgrid cluster control system is greater than a second threshold, the purchase price of electricity for the microgrid cluster control system is determined by the ratio of the sum of the absolute value of the net output power of the microgrid cluster control system and a second product to the net demand power of the microgrid cluster control system, where the second product is determined by the product of the electricity exchange price of the microgrid and the remaining energy of the microgrid cluster control system; if the remaining energy of the microgrid cluster control system is equal to or less than the second threshold, the purchase price of electricity for the microgrid cluster control system is determined based on the electricity exchange price of the microgrid.

[0038] This implementation combines multiple parameters, including the microgrid cluster control system's surplus power, net output power, net demand power, and the electricity exchange price between the microgrid and the battery swapping station, to construct a dynamic price adjustment mechanism. This enables the microgrid cluster control system's electricity purchase and sales prices to better reflect the actual energy supply and demand situation. For example, when the microgrid cluster control system's surplus energy falls within different ranges, different calculation logics are used to determine the price. This considers both the local energy exchange price between the battery swapping station and the microgrid, as well as the overall power balance needs of the cluster. This allows for precise guidance of energy flow through price signals, promoting coordinated and optimized operation among the various entities within the microgrid cluster.

[0039] S220, based on the first interaction parameter, determine the first local observation data of the microgrid at the current moment.

[0040] The first local observation data of a microgrid at the current moment refers to the set of various parameters that can be collected or calculated by the microgrid under its current operating state. It is used to reflect the real-time energy supply and demand characteristics, equipment operating status and external interaction environment of the microgrid.

[0041] Determining the first local observation data of the microgrid at the current moment based on the first interaction parameter includes: using the first interaction parameter as the first local observation data of the microgrid at the current moment; or, determining other first interaction parameters between the microgrid and the battery swapping station at the current moment based on the first interaction parameter, such as the exchange power between the microgrid and the battery swapping station at the current moment; or determining the first local observation data of the microgrid at the current moment based on other first interaction parameters.

[0042] If a third interaction parameter is obtained from the microgrid cluster control system, then the first local observation data of the microgrid at the current moment is determined based on the first interaction parameter, including: determining the first local observation data of the microgrid at the current moment based on the first interaction parameter and the third interaction parameter.

[0043] In some implementations, the first local observation data of the microgrid at the current moment is determined based on the first interaction parameter, including: determining the first local observation data of the microgrid at the current moment based on the first interaction parameter, the third interaction parameter, and the operating parameters of the microgrid at the current moment.

[0044] For example, the operating parameters of a microgrid at the current moment may include at least one of photovoltaic output power, wind turbine output power, and load power.

[0045] S230, based on the first local observation data, uses the policy network to determine the first scheduling action data of the microgrid at the current moment.

[0046] The first scheduling action data is the scheduling action data of the microgrid at the current moment, determined by the policy network. For example, the first scheduling action data includes second interaction parameters. The second interaction parameters are also interaction parameters between the battery swapping station and the microgrid, but reflect the microgrid's pricing strategy for energy exchange with the battery swapping station. For example, the second interaction parameters include the electricity exchange price between the microgrid and the battery swapping station at the current moment.

[0047] In some implementations, the logic for determining the electricity exchange price between the microgrid and the battery swapping station at the current moment can be as follows: Based on the exchange power between the microgrid and the battery swapping station at the current moment, and the output power of renewable energy in the microgrid at the current moment, determine the total power of the microgrid at the current moment; based on the total power of the microgrid at the current moment, determine the electricity exchange price between the microgrid and the battery swapping station at the current moment. For example, renewable energy includes wind power generation based on wind turbines, photovoltaic power generation, etc.

[0048] In some implementations, determining the power exchange price between the microgrid and the battery swapping station based on the total power of the microgrid includes: if the total power of the microgrid is greater than or equal to a reference power of the microgrid, then determining the power exchange price between the microgrid and the battery swapping station based on the ratio of the total power of the microgrid to the reference power, where the reference power corresponds to the load baseline level of the microgrid; if the total power of the microgrid is less than the reference power of the microgrid, then determining the power exchange price between the microgrid and the battery swapping station based on the ratio of a first difference to the reference power of the microgrid, where the first difference is determined based on the difference between the output power of renewable energy in the microgrid and the exchange power between the microgrid and the battery swapping station.

[0049] This implementation constructs a differentiated pricing mechanism by comparing the total power of the microgrid with a reference power. When the total power is higher than the reference power, it indicates that the current energy supply of the microgrid is relatively sufficient. At this time, the electricity exchange price is determined based on the ratio of total power to reference power, which can guide the battery swapping stations to increase the amount of electricity purchased during periods of abundant energy, thereby improving the renewable energy absorption rate within the microgrid. Conversely, when the total power is lower than the reference power, the price is calculated based on the difference between the renewable energy output power and the exchanged power. This allows for reasonable adjustment of the battery swapping stations' electricity consumption behavior during periods of energy shortage, ensuring a stable supply to the microgrid's own load. This method of dynamically adjusting prices based on the real-time power status of the microgrid achieves refined allocation of energy resources, taking into account both the operational economy of the microgrid and providing accurate price signals for the energy interaction decisions of battery swapping stations.

[0050] Optionally, the first scheduling action data may also include at least one of the following parameters: the active power of the diesel engine, the discharge power of the energy storage battery, the charging power of the energy storage battery, the load power reduction ratio, and the exchange power between the microgrid and the microgrid cluster control system.

[0051] In this embodiment of the application, the multi-agent framework corresponding to the policy network includes microgrids and battery swapping stations as different agents. The local observations of the microgrid agent include the first local observation data of the microgrid, the actions include the first scheduling action data of the microgrid, and the reward function includes the negative cost function of the microgrid.

[0052] This application's embodiments introduce a multi-agent near-end policy optimization algorithm through a policy network, enabling the technical solution of this application's embodiments to simultaneously handle distributed energy fluctuations and load changes on the microgrid side and battery supply and demand dynamics on the battery swapping station side, thereby achieving joint optimization of multi-agent decision-making.

[0053] In some implementations, the cost function of a microgrid can be determined by: determining the cost function of the microgrid based on at least one of the following: electricity trading cost, generation cost, emission cost, and power reduction cost. The operational constraints of a microgrid can be determined by: determining the operational constraints based on at least one of the following: power balance constraints between the microgrid and the battery swapping station and the microgrid cluster control system; power constraints of renewable energy sources in the microgrid; charging and discharging power constraints of energy storage batteries in the microgrid; maximum and minimum exchange power constraints of energy storage batteries in the microgrid; maximum and minimum exchange power constraints of the battery swapping station; maximum and minimum exchange power constraints of the microgrid cluster control system; load reduction constraints; energy consumption constraints of energy storage stations in the microgrid; and voltage constraints in the microgrid.

[0054] To achieve stable training and efficient iteration of the policy network, in some implementations, after determining the first scheduling action data of the microgrid at the current moment using the policy network based on the first local observation data, the interaction method further includes: sending second interaction parameters to the battery swapping station; receiving the updated first interaction parameters determined by the battery swapping station based on the second interaction parameters, the updated first interaction parameters including the number of charging batteries and / or discharging batteries at the battery swapping station at the current moment; and updating the first scheduling action data of the microgrid at the next moment using the policy network based on the updated first interaction parameters. Based on the first local observation data and the first scheduling action data of the microgrid at the current moment, the first reward value of the microgrid at the current moment is determined; the first local observation data, the first scheduling action data, and the first reward value at the current moment are stored in a first database. The first database stores multiple sets of first training samples, each set of first training samples including first historical local observation data, first historical scheduling action data, and the corresponding first historical reward value at any historical moment, and the policy network is trained using the first training samples.

[0055] The updated first scheduling action data includes the updated second interaction parameters, which include the electricity exchange price between the microgrid and the battery swapping station at the next time step.

[0056] The policy network is obtained by: determining the loss value of the policy network based on the first training sample; and updating the network parameters of the policy network based on the loss value of the policy network.

[0057] In some implementations, determining the loss value of the policy network based on the first training sample may include: if the number of the first training samples in the first database reaches a certain threshold, then using the first training samples with high reward values ​​in the first database to determine the loss value of the policy network.

[0058] Optionally, determining the loss value of the policy network based on the first training samples may further include: determining the loss value of the policy network at the first historical moment based on the first training samples at the first historical moment and the next historical moment; and updating the network parameters of the policy network based on the loss value of the policy network at at least one historical moment. Here, at least one historical moment includes the aforementioned first historical moment.

[0059] In this embodiment, during the process of determining the first scheduling action data using the policy network, the first local observation data of the microgrid at different times, the corresponding first scheduling action data, and the first reward value are stored in a first database. When the number of samples in the first database reaches a threshold, the first training samples with high reward values ​​are extracted from the first database, and these high-reward-value sample data are used to update the network parameters of the policy network. This embodiment guides the policy network to gradually learn a better scheduling strategy through the first reward value, so as to maximize the overall benefits of collaborative interaction with the battery swapping station while ensuring the stable operation of the microgrid itself. In the process of updating the policy network parameters, this embodiment can also adopt an adaptive learning rate adjustment mechanism, which dynamically adjusts the learning rate according to the changing trend of the network loss function, avoiding oscillations or slow convergence during the training process, and further improving the learning efficiency and decision accuracy of the policy network.

[0060] Figure 3 The diagram shown is a flowchart illustrating another interaction method between a microgrid and a battery swapping station provided in an embodiment of this application. See also... Figure 3 This application provides another method for interaction between a microgrid and a battery swapping station. The method is applied to the battery swapping station, which is connected to the microgrid. The interaction method includes the following steps.

[0061] S310 receives the second interactive parameters sent by the microgrid.

[0062] The second interaction parameter includes the electricity exchange price between the microgrid and the battery swapping station at the current moment.

[0063] S320, based on the second interaction parameter, determines the second local observation data of the battery swapping station at the current moment.

[0064] The second local observation data of a battery swapping station at the current moment refers to the set of various parameters related to energy interaction that can be collected or calculated by the battery swapping station under its current operating state. These parameters are used to reflect the battery supply and demand situation, charging load characteristics, and price interaction status with the microgrid.

[0065] Determining the second local observation data of the battery swapping station at the current time based on the second interaction parameter includes using the second interaction parameter as the second local observation data of the battery swapping station at the current time, or determining other second interaction parameters between the microgrid and the battery swapping station at the current time based on the second interaction parameter, and determining the second local observation data of the battery swapping station at the current time based on the other second interaction parameters.

[0066] In some implementations, determining the second local observation data of the battery swapping station at the current moment based on the second interaction parameter may also include: determining the second local observation data of the battery swapping station at the current moment based on the second interaction parameter and the operating parameters of the battery swapping station at the current moment.

[0067] For example, the operating parameters of a battery swapping station at the current moment may include at least one of the following: the number of electric vehicles arriving, the number of backlogged electric vehicles that have not completed their battery swapping requirements within the time interval, the number of fully charged batteries, and the number of depleted batteries.

[0068] S330, based on the second local observation data, uses the main network to determine the Q value of each scheduling action of the battery swapping station.

[0069] The Q-value of each scheduling action measures the expected cumulative reward that a battery swapping station can obtain by executing different scheduling actions under the second local observation data, reflecting the long-term value of each scheduling action. The main network constructs a mapping relationship between the second local observation data and the Q-value of scheduling actions by learning from historical interaction data, providing a quantitative basis for the decision-making of the battery swapping station.

[0070] In this embodiment of the application, the multi-agent framework corresponding to the main network includes microgrids and battery swapping stations as different agents. The local observations of the battery swapping station agent include the second local observation data of the battery swapping station, the actions include the second scheduling action data of the battery swapping station, and the reward function includes the profit function of the battery swapping station.

[0071] This application embodiment combines a Q-network with deep reinforcement learning in the main network, enabling battery swapping stations to dynamically optimize their scheduling strategies based on the second interaction parameters sent by the microgrid and their own operating status. For example, when the electricity price of the microgrid is low, the main network may output a scheduling action with a high Q value, such as "increasing the number of charging batteries," to incentivize battery swapping stations to store more electricity during low-price periods. Conversely, when prices rise to high levels, the main network may prioritize actions such as "reducing charging volume and prioritizing emergency battery swapping needs," thereby balancing battery swapping profits and service efficiency.

[0072] In some implementations, the profit function of a battery swapping station is determined by: determining the profit function of the battery swapping station based on at least one of the following: battery exchange profit between the battery swapping station and the microgrid, power exchange profit between the battery swapping station and the microgrid, backlog penalty cost of the battery swapping station, and battery aging cost of the battery swapping station.

[0073] The determination of the operational constraints of a battery swapping station includes at least one of the following: the quantitative relationship between the number of fully charged batteries in the swapping station, the number of depleted batteries and the total number of batteries in the swapping station, the maximum number of batteries that can be charged and discharged in the swapping station, the minimum and maximum power range constraints during battery charging, and the battery capacity constraints.

[0074] S340 generates the second scheduling action data of the battery swapping station at the current moment based on the Q value of each scheduling action of the battery swapping station.

[0075] The second scheduling action data includes the updated first interaction parameters, which include the number of charging batteries and / or discharging batteries at the current time of the battery swapping station.

[0076] In some implementations, the second scheduling action data of the battery swapping station at the current moment is generated based on the Q value of each scheduling action of the battery swapping station. This includes: generating a random number; if the random number is less than a probability threshold, then randomly selecting an action from each scheduling action to generate the second scheduling action data of the battery swapping station at the current moment; if the random number is greater than or equal to the probability threshold, then selecting the action with the largest Q value from each scheduling action of the battery swapping station to generate the second scheduling action data of the battery swapping station at the current moment.

[0077] The technical solution of this application embodiment achieves a balance between exploration and utilization in the scheduling actions of battery swapping stations by introducing an ε-greedy strategy. Here, the probability threshold corresponds to the ε value, which is used to control the probability of random exploration. This mechanism of dynamically adjusting the exploration and utilization ratio can effectively balance the innovative attempts and experience accumulation of battery swapping stations during the strategy learning process, ensuring that the main network can quickly converge to a near-optimal scheduling strategy.

[0078] To achieve continuous optimization of the main network and dynamic iteration of scheduling strategies, in some implementations, after generating the second scheduling action data of the battery swapping station at the current moment based on the Q value of each scheduling action of the station, the interaction method further includes: sending the updated first interaction parameters to the microgrid; receiving the updated second interaction parameters determined by the microgrid based on the updated first interaction parameters, the updated second interaction parameters including the electricity exchange price between the microgrid and the battery swapping station at the next moment; and updating the second scheduling action data of the battery swapping station at the next moment using the main network based on the updated second interaction parameters, the second scheduling action data of the battery swapping station at the next moment including the number of charging batteries and / or discharging batteries of the battery swapping station at the next moment.

[0079] In some implementations, after generating the second scheduling action data of the battery swapping station at the current moment based on the Q values ​​of each scheduling action of the station, the interaction method further includes: determining the second reward value of the battery swapping station at the current moment based on the second local observation data and the second scheduling action data at the current moment; and storing the second local observation data, the second scheduling action data, and the second reward value at the current moment in a second database. The first database can be the same as or different from the first database mentioned above, depending on actual needs.

[0080] In some implementations, the second database stores multiple sets of second training samples. Each set of second training samples includes second historical local observation data, second historical scheduling action data, and corresponding second historical reward values ​​at any historical moment. The main network obtains these values ​​through the following methods: based on the second training samples, the target network determines the first Q value of the second training samples; based on the second training samples, the main network determines the second Q value of the second training samples; based on the first Q value and the second Q value, the loss value of the main network is determined; and based on the loss value of the main network, the network parameters of the main network are updated.

[0081] The target network is used to generate a relatively fixed target Q value to avoid fluctuations in the target value caused by frequent updates of the main network parameters, while the main network optimizes its parameters by continuously learning the difference between the target Q value and its own output Q value.

[0082] The technical solution of this application embodiment effectively improves the stability of the training process through a dual-network architecture design of a main network and a target network.

[0083] This application also provides an interaction method between a microgrid and a power switching station, which is applied to... Figure 1 The energy interaction system architecture shown includes a microgrid consisting of wind turbines, energy storage stations, photovoltaic panels, loads, and diesel generators. Each swapping station comprises a swapping station controller, battery charging / discharging stations, electric vehicle swapping stations, and a battery storage warehouse. Microgrid controllers and swapping station controllers are deployed at both the microgrid and swapping station levels, forming a two-layer control architecture. The optimization objective of this two-layer architecture is to minimize the operating costs of the microgrid at the microgrid level and maximize the profits of the swapping stations at the swapping level. At the microgrid level, the microgrid controller optimizes internal decisions and sets personalized electricity prices for the swapping stations. At the swapping station level, the swapping station controller responds to this price, makes operational decisions for the swapping station, and then sends the optimized charging and discharging plan back to the microgrid while meeting the station's operational constraints. Through iterative optimization, a win-win situation is achieved for both the microgrid and the swapping stations.

[0084] In some implementations, the objective function of the microgrid control model in a two-layer control architecture is as follows:

[0085] minC MG =min(C tra +C gen +C emi +C de,load (1)

[0086]

[0087] Where T is the scheduling period. It is the electricity exchange price provided by the microgrid to the battery swapping station at time t. The purchase and sale price of electricity for a microgrid cluster control system. P represents the transaction revenue obtained by microgrid i in the microgrid cluster control system. t bss ,P t com These are the exchange power between the microgrid and the battery swapping station, and the exchange power between the microgrid and the microgrid cluster control system. t bss ,P t com A positive value indicates the purchase of electricity from a microgrid; otherwise, a negative value indicates the purchase of electricity from a microgrid. It is the active power of the diesel engine m={1,2,...,M} at time t. N is the generator cost coefficient for diesel engines. gas It is the quantity of pollutant types, ρ j σ is the unit quality control cost of pollutant j. mj ρ is the emission coefficient of pollutant j in diesel engine m. n,t Let N be the unit power reduction cost of load n at time t. load σ represents the total number of loads. n,t The load power reduction ratio, Let be the power of load n at time t.

[0088] The operating constraints of microgrids are as follows:

[0089]

[0090] 0≤σ n,t ≤σ n,max (10)

[0091]

[0092] V l,min ≤V l,t ≤V l,max (13)

[0093] in, These represent the output power of the photovoltaic power unit p and the wind turbine power unit ω, respectively. This refers to the exchange power between the microgrid and the battery swapping station, as well as the exchange power between the microgrid and the microgrid cluster control system. For the charging and discharging power of the energy storage battery, N pv N wt N ess These are collections of photovoltaic, wind turbine, and energy storage power stations. This represents the maximum power output of photovoltaic and wind turbines. Let be the maximum charging and discharging power of energy storage power station s at time t. The maximum charging and discharging power of the energy storage power station. These are the maximum and minimum switching power of the battery swapping station. σ represents the maximum and minimum switching power of the microgrid cluster control system. n,max This is the maximum load reduction factor. Let be the amount of electricity stored in energy storage station s at time t. V represents the maximum and minimum power capacity of the energy storage power station. g,t - is the node voltage of g, V l,t Vl is the voltage of bus l at time t, V1 is the bus reference voltage, and rl is the voltage of bus l at time t. gl ,ω gl For the line resistance and reactance gl, P gl,t Q gl,t For the active and reactive power flow of line gl, V l,max V l,min The maximum and minimum voltages of bus l.

[0094] The objective function of the battery swapping station control model is shown below:

[0095]

[0096]

[0097] in, For the battery swapping profit of the battery swapping station, δ ev It's a fixed service fee for replacing each battery. This is the price per kilowatt-hour for battery swapping. It refers to the number of electric vehicles in service. These represent the battery's charge level when it's depleted and fully charged, respectively. For the profit from power exchange between battery swapping stations and microgrids, here This refers to the number of batteries being discharged / recharged. P t dch / P t ch This is the discharge / charge power of a single battery. If P t bss>0, the battery swapping station discharges to the microgrid. This represents the revenue of the battery swapping station. Conversely, if P t bss <0, the battery swapping station charges from the microgrid. This represents the cost of charging for the microgrid. To account for the uncertainty of electric vehicle arrivals, embodiments of this application also introduce backlog penalty costs. in It is the number of unsold electric vehicles, δ pen It is the penalty coefficient. The cost of battery aging within the battery swapping station is calculated, taking into account both cycle aging and calendar aging. C is the maximum number of batteries in a battery swapping station. cyc C represents the battery capacity loss per second. t is n t The capacity decrease was measured experimentally after each cycle, with each cycle lasting t1 seconds. Coefficient F i-DoD =k1(1-DoD)+k2, Related to depth of discharge (DoD) and battery temperature (T) respectively bat Charging rate C rate-c and discharge rate C rate-d Related. Coefficients k1 to k8 are determined based on the battery model used, and the battery aging cost C. cal Only affected by temperature T bat The influence of the coefficients h1 to h3 is determined based on the battery.

[0098] The operational constraints of the battery swapping station are as follows:

[0099]

[0100] in This indicates the number of fully charged and depleted batteries, as well as the total number of batteries at the battery swapping station. These represent the number of batteries that can be charged or discharged at the battery swapping station. P is the maximum number of batteries that can be charged and discharged at a battery swapping station. t ch P t dch These are the battery charging power and the battery discharging power, respectively. Minimum and maximum charging power for the battery. Minimum and maximum battery discharge power. Let t be the battery charge at time t. This is the initial battery charge.

[0101] Considering distributed energy resources, energy pricing, the randomness of electric vehicle arrival times, and penalties for backlogged electric vehicles, this application also proposes a pricing scheme for a microgrid cluster control system with intermediate market profits to balance the supply and demand of the microgrid-battery swapping station system.

[0102] First, considering the supply and demand relationship of electricity within the microgrid, a real-time pricing scheme based on demand response is proposed. Then, after meeting the needs of local load and battery swapping stations, the microgrid adopts a mid-market rate pricing scheme to trade surplus / shortage energy with other microgrids in the microgrid cluster control system.

[0103] For microgrid-based electricity pricing schemes that rely on real-time dispatch, the following conditions must be met:

[0104] 1) During charging and discharging, the electricity exchange price between the battery swapping station and the microgrid depends on the total load of the microgrid. This load is determined by the equivalent load of the microgrid and the algebraic sum of the electricity exchanged between the battery swapping station and the microgrid.

[0105] 2) If the total load level is greater than or equal to the equivalent load of the microgrid, the price should be greater than or equal to the reference price; otherwise, it should be lower than the reference price.

[0106] Electricity exchange price between microgrids and battery swapping stations The description is as follows:

[0107]

[0108] Among them, P t el It is the equivalent load of a microgrid, defined as the negative value of the difference between the power output of renewable energy and the load. λ represents the predefined equivalent load reference power. rf It is a constant representing a reference price.

[0109] For the power purchase and sale price of a microgrid cluster control system, let P... t com,ng ,P t com,nd Let P represent the net power output and net demand of the microgrid cluster control system at time t, respectively. t com,r As the net load P of the microgrid t nl The function.

[0110]

[0111] The local purchase price of microgrid cluster control systems and local sales price The calculation formula is as follows:

[0112]

[0113] in, That is Therefore, within time period t, the transaction revenue obtained by microgrid i in the microgrid cluster control system It can be calculated as follows:

[0114]

[0115] in, for for

[0116] The microgrid and switching station interaction method provided in this application embodiment also applies an algorithm architecture that combines a multi-agent near-end policy optimization algorithm with a dual-deep Q-network optimization algorithm. Figure 4 The diagram illustrates an algorithm architecture provided in an embodiment of this application. The microgrid controller and the battery swapping station controller interact within this architecture. In this algorithm architecture, the embodiment of this application formulates the two-layer control problem of the microgrid and battery swapping station as a partially observable Markov game. It combines a multi-agent near-end policy optimization algorithm and a dual-depth Q-network optimization algorithm, employing a hybrid algorithm that solves the mixed-integer nonlinear problem without requiring discretization or continuous transformation, and is used for optimal scheduling decisions of the microgrid and battery swapping station.

[0117] Regarding the transformation of Markov games, in the interactive energy management problem between microgrids and battery swapping stations, the profits of the microgrids and battery swapping stations depend on the decisions of themselves and other participants. Therefore, the embodiments of this application formalize the two-level decision problem into a partially observable Markov decision. Based on the input state, each agent obtains local measurement data, and then adopts a strategy based on its local measurement data to select the action to be executed and transition to the next state; this process is based on a transition function. At each time step t, agent i, based on its private observation o... i,t Take action a i,t Then return the instant reward r i,t And a new private observation o i,t+1 The goal of each agent i is to maximize its total expected reward by learning an optimal policy, which is expressed as:

[0118]

[0119] Where, γ t ∈[0,1) represents the discount factor, Π i Let represent the set of all policies of agent i, and Let represent the optimal policy of other agents, and ES be the expectation.

[0120] In this embodiment, each microgrid and swapping station is treated as a different intelligent agent, and the agent is defined for local observation, action response, and reward function.

[0121] For example, at time t, the microgrid controller MG i Observations of the controller i,t The definition is as follows (32). Based on the current supply and demand situation of the microgrid and the power exchange station, as well as the purchase price and sales price of the microgrid cluster control system, MG i It will make internal operational decisions to minimize its operating costs.

[0122]

[0123] in, The numbers represent the output power of photovoltaic power, the output power of wind turbine, the power of the load, the purchase and sale price of electricity in the microgrid cluster control system, the electricity exchange price between the microgrid and the battery swapping station, and the exchange power between the microgrid and the battery swapping station, respectively.

[0124] Based on the (36) microgrid controller MG i action a i,t The calculation is based on the exchange power of the microgrid and microgrid cluster control system, the output power of the diesel engine, the charging and discharging power of the energy storage battery, and the flexible load reduction ratio.

[0125]

[0126] in, These represent the diesel engine output power, the energy storage battery discharge power, the energy storage battery charging power, the flexible load reduction ratio, the electricity exchange price between the microgrid and the battery swapping station, and the exchange power between the microgrid and the microgrid cluster control system, respectively.

[0127] At time t, the local observation value of the battery swapping station It can be defined as follows:

[0128]

[0129] in, Indicates the number of electric vehicles arriving. This represents the number of backlogged electric vehicles that have not completed their battery replacement requirements within time interval t. ) represent the number of fully charged batteries and the number of depleted batteries in the battery swapping station, respectively.

[0130] At time t, the operation of the battery swapping station Defined as follows:

[0131]

[0132] in, This indicates the number of electric vehicles participating in the battery swapping service.

[0133] According to equations (39) and (40), the reward function of the microgrid agent is... Reward function of the agent of the battery swapping station The definition is as follows:

[0134]

[0135] Among them, C tra C gen C emi C de,load These represent the electricity trading costs, generation costs, emission costs, and power reduction costs of a microgrid, respectively. For the profit of battery swapping stations and microgrids, For the profit of power exchange between battery swapping stations and microgrids, The penalty cost for the backlog of batteries waiting to be swapped at the battery swapping station. This refers to the cost of battery aging at battery swapping stations.

[0136] Based on the above algorithm architecture, the microgrid controller makes internal operation scheduling decisions and provides prices to the battery swapping stations. Based on this price, the battery swapping station controller meets the battery swapping needs of electric vehicles and provides feedback on charging / discharging plans. Both controllers use each other's action responses as inputs to their measurements. The information interaction between the multi-agent proximal policy optimization network and the dual-deep Q-network is as follows: Figure 5 As shown. Similarly, the battery swapping station controller observes the microgrid's operation, the battery swapping demand of electric vehicles, and their local battery utilization status to obtain local measurement data for the battery swapping station. See also Figure 4 and Figure 5 Based on local observation data from microgrids and battery swapping stations, the method for realizing interaction between microgrids and battery swapping stations includes the following steps.

[0137] 1) First, update the network parameters θ of the evaluation network in the microgrid. c And the network parameters θ of the battery swapping station network, i.e., the main network of the battery swapping station. b And update the network parameters θ of the old policy network. old The network parameters θ of the new strategy network are obtained. new .

[0138] 2) Microgrid controller based on the number of rechargeable batteries in the battery swapping station Number of batteries in the battery swapping station And local input data of the microgrid obtained by local measurement Where b represents different battery swapping stations, t represents time t, and i represents different microgrids.

[0139] 3) The microgrid controller will input local data The input network parameter is θ new The new strategy network obtains its device scheduling action data.

[0140] 4) The microgrid controller performs its scheduling actions. And obtain its reward function and local input data Data group Stored in historical database data.

[0141] 5) The battery swapping station controller is based on the electricity exchange price between the microgrid and the battery swapping station. Electric vehicle demand and local conditions are obtained through local input data.

[0142] 6) The battery swapping station controller will input local data. The input network parameter is θ b The network of battery swapping stations is used to obtain the scheduling actions of the battery swapping stations through an ε-greedy strategy. Where P g ε represents the generated random number, and ε represents the probability threshold.

[0143] 7) The battery swapping station controller performs its scheduling actions. And obtain its reward function and the local input data at the next moment Data group Stored in historical database data.

[0144] After a fixed iteration cycle, a batch of data is sampled from the historical database to train the network parameters of each network in the microgrid and battery swapping station. See also... Figure 4 and Figure 5 The training process for the policy network and evaluation network in a microgrid is as follows:

[0145] 1) When the historical data in the microgrid's historical database meets the training conditions, a set number of historical data are extracted according to priority, and the generalized favorable factors are calculated based on equation (41).

[0146]

[0147] Where V is the network parameter θ of the evaluation network. c and the network parameters θ of the policy network new The parameterized value function, where φ represents the equilibrium partial variance and γ represents the coefficient.

[0148] 2) Calculate the evaluation loss value L(θ) using equation (42). c Then, by combining equation (42) with the behavioral constraints of the microgrid, the network parameters θ of the evaluation network in the microgrid are updated. c .

[0149]

[0150] 3) Calculate the action response loss value L(θ) using equations (43) and (44). new Then, by combining equation (43) with the operating constraints of the microgrid, the network parameters θ of the microgrid strategy network are updated. new .

[0151]

[0152] Where ε = [0,1] is a random parameter, rat t (θ) is the ratio between the new policy network and the old policy network, used to prevent over-updating of the policy, and its range is [1-ε, 1+ε].

[0153] 4) After updating the network parameters of the microgrid evaluation network and the strategy network a certain number of times, the network parameters θ of the old strategy network in the old microgrid are... old Update the network parameters θ of the new policy network new And by updating the selection probability The selection priority for updating historical data of microgrids is determined, where K is a coefficient.

[0154] See also Figure 4 and Figure 5 The training process for the battery swapping station network and the target network in the battery swapping station is as follows:

[0155] 1) When the historical data in the historical database of the battery swapping station meets the training conditions, a set number of historical data are extracted according to priority. The network parameters θ of the target network are calculated by combining equation (45) with the operating constraints of the battery swapping station. Tb .

[0156]

[0157] in, It is the reward function for the battery swapping station.

[0158] 2) Calculate the loss value L(θ) of the battery swapping station network using equation (46). b By combining equation (46) with the operational constraints of the battery swapping station, network losses are minimized, and the network parameters θ of the battery swapping station network are updated. b .

[0159]

[0160] 3) After updating the network parameters of the target network of the battery swapping station and the network parameters of the battery swapping station network a certain number of times, the network parameters θ of the target network of the battery swapping station will be updated. Tb Update the network parameters θ to reflect the current battery swapping station network. b And update the historical data of the battery swapping station to select priority.

[0161] The technical solution of this application proposes a hybrid algorithm combining a multi-agent near-end policy optimization algorithm and a dual-deep Q-network optimization algorithm to coordinate the sequential decision-making between the microgrid and the battery swapping station. Compared with deep reinforcement learning methods, this reduces the operating cost of the microgrid and increases the profit of the battery swapping station. Furthermore, through a microgrid cluster control system with a pricing scheme based on the intermediate market interest rate, economic interaction between the microgrid and the battery swapping station is achieved while maintaining system voltage stability. In addition, this application's embodiment adopts a two-layer energy management architecture for the microgrid and the battery swapping station, ensuring the economic benefits of the battery swapping station and the cost of the microgrid, while enabling the battery swapping station to quickly respond to the peak-shaving and valley-filling needs of the microgrid and maintaining a low electric vehicle backlog.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for interaction between a microgrid and a battery swapping station, characterized in that, Applied to the microgrid, which is connected to the battery swapping station, the interaction method includes: Obtain the first interaction parameters sent by the battery swapping station, the first interaction parameters including the number of charging batteries and / or discharging batteries of the battery swapping station at the previous moment; Based on the first interaction parameters, the first local observation data of the microgrid at the current moment is determined; Based on the first local observation data, the first scheduling action data of the microgrid at the current time is determined using the policy network. The first scheduling action data includes a second interaction parameter, which includes the electricity exchange price between the microgrid and the battery swapping station at the current time.

2. The method according to claim 1, characterized in that, After determining the first scheduling action data of the microgrid at the current time using the policy network based on the first local observation data, the interaction method further includes: Send the second interaction parameter to the battery swapping station; The system receives updated first interaction parameters determined by the battery swapping station based on the second interaction parameters. The updated first interaction parameters include the number of charging batteries and / or discharging batteries of the battery swapping station at the current time. Based on the updated first interaction parameters, the policy network is used to update the first scheduling action data of the microgrid at the next time step. The updated first scheduling action data includes the updated second interaction parameters, which include the electricity exchange price between the microgrid and the battery swapping station at the next time step.

3. The method according to claim 1, characterized in that, After determining the first scheduling action data of the microgrid at the current time using the policy network based on the first local observation data, the interaction method further includes: Based on the first local observation data of the microgrid at the current time and the first scheduling action data at the current time, the first reward value of the microgrid at the current time is determined; The first local observation data, the first scheduling action data, and the first reward value at the current moment are stored in the first database.

4. The method according to claim 3, characterized in that, The first database stores multiple sets of first training samples. Each set of first training samples includes first historical local observation data, first historical scheduling action data, and corresponding first historical reward value at any historical moment. The policy network is obtained through the following method: Based on the first training sample, determine the loss value of the policy network; The network parameters of the policy network are updated based on the loss value of the policy network.

5. The method according to claim 1, characterized in that, The microgrid is connected to a microgrid cluster control system, which is connected to a main power grid. The method further includes: The third interaction parameter sent by the microgrid cluster control system is obtained, and the third interaction parameter includes the electricity purchase price and electricity sales price of the microgrid cluster control system at the current time. The step of determining the first local observation data of the microgrid at the current moment based on the first interaction parameters includes: Based on the first interaction parameter and the third interaction parameter, the first local observation data of the microgrid at the current time is determined.

6. A method for interaction between a microgrid and a battery swapping station, characterized in that, Applied to the battery swapping station, which is connected to the microgrid, the interaction method includes: Receive a second interaction parameter sent by the microgrid, the second interaction parameter including the electricity exchange price between the microgrid and the battery swapping station at the current time; Based on the second interaction parameter, the second local observation data of the battery swapping station at the current moment is determined; Based on the second local observation data, the Q value of each scheduling action of the battery swapping station is determined using the main network; Based on the Q value of each scheduling action of the battery swapping station, a second scheduling action data of the battery swapping station at the current time is generated. The second scheduling action data includes an updated first interaction parameter, which includes the number of charging batteries and / or discharging batteries of the battery swapping station at the current time.

7. The method according to claim 6, characterized in that, After generating the second scheduling action data of the battery swapping station at the current time based on the Q values ​​of each scheduling action of the battery swapping station, the interaction method further includes: The updated first interaction parameters are sent to the microgrid; The microgrid receives an updated second interaction parameter determined based on the updated first interaction parameter, wherein the updated second interaction parameter includes the electricity exchange price between the microgrid and the battery swapping station at the next time step. Based on the updated second interaction parameters, the main network is used to update the second scheduling action data of the battery swapping station at the next time moment. The second scheduling action data of the battery swapping station at the next time moment includes the number of charging batteries and / or discharging batteries of the battery swapping station at the next time moment.

8. The method according to claim 6, characterized in that, After generating the second scheduling action data of the battery swapping station at the current time based on the Q values ​​of each scheduling action of the battery swapping station, the interaction method further includes: Based on the second local observation data of the battery swapping station at the current time and the second scheduling action data at the current time, the second reward value of the battery swapping station at the current time is determined; The second local observation data, the second scheduling action data, and the second reward value at the current moment are stored in the second database.

9. The method according to claim 6, characterized in that, The second database stores multiple sets of second training samples. Each set of second training samples includes second historical local observation data, second historical scheduling action data, and a corresponding second historical reward value at any historical moment. The main network obtains these data through the following method: Based on the second training sample, the first Q value of the second training sample is determined using the target network, and the network parameters of the target network are periodically copied from the network parameters of the main network. Based on the second training sample, the second Q value of the second training sample is determined using the main network; Based on the first Q value and the second Q value, the loss value of the main network is determined; The network parameters of the main network are updated based on the loss value of the main network.

10. The method according to claim 6, characterized in that, The step of generating second scheduling action data for the battery swapping station at the current moment based on the Q values ​​of each scheduling action of the battery swapping station includes: Generate a random number; If the random number is less than the probability threshold, then one action is randomly selected from the scheduling actions to generate the second scheduling action data of the battery swapping station at the current time; If the random number is greater than or equal to the probability threshold, then the action with the largest Q value is selected from the scheduling actions of the battery swapping station to generate the second scheduling action data of the battery swapping station at the current time.

Images