Energy Internet transaction method and system based on reinforcement learning blockchain empowerment

Abstract

The present invention discloses an energy internet transaction method and system based on reinforcement learning blockchain empowerment, comprising the following steps: constructing a three-stage game model between operators, retailers and producers and sellers based on the energy transaction relationship between operators, retailers and producers and sellers on the blockchain transaction platform; using a distributed hierarchical policy gradient algorithm to solve the game equilibrium point in the three-stage game model, the game equilibrium point includes the best unit service price, the best unit energy price and the best energy demand; operators, retailers and producers and sellers conduct energy transactions according to the game equilibrium point. The present invention can help operators and retailers achieve higher utility, while producers and sellers can also achieve better utility.

Description

Technical Field

[0001] The present invention belongs to the technical field of energy internet, and specifically relates to an energy internet transaction method and system based on reinforcement learning blockchain empowerment. Background Art

[0002] With the development trend of distributed energy, Energy Internet (EI) has quickly become the focus of attention. However, due to the intermittent and uncertain nature of distributed energy, the influx of a large number of distributed energy and traditional control methods have hindered the development of Energy Internet. At the same time, the emergence of software defined network (SDN) has brought reliability and flexibility to solve these problems. Due to reasonable prices and effective transmission, the distributed energy market has gradually formed in the Energy Internet. It enables traditional energy consumers to transform into energy retailers with the ability to produce, store and sell distributed energy. This model can reduce transmission losses and reduce the load peak of Energy Internet.

[0003] On the other hand, under the general trend of the Internet of Things, edge computing has been widely used in various network computing architectures due to its advantages in network latency, scalability and reliability. In order to continuously provide reliable computing, storage and communication services, the energy utilization and supply of edge servers, gateways and other devices are issues that need to be explored urgently.

[0004] In order to simultaneously meet the energy needs of emerging energy retailers and various edge devices in the energy Internet, an energy trading market serving edge computing is proposed. Although this model can effectively meet the needs of both parties, there are still many problems that need to be solved: 1) The credit crisis between different trading entities makes it impossible to reliably conduct energy trading; (2) Imperfect market modeling: The models of various roles in the current energy trading market are not perfect, and a trading process with mutual interaction and constraints has not been formed; (3) Unbalanced utility: The current optimization mechanism in energy trading focuses on maximizing the utility of one party, without considering the utility balance between multiple parties. In addition, in order to achieve the situation of utility balance, most current studies use game theory methods to simulate the interaction between parties in the transaction process. Traditional methods usually assume that there is a centralized organization to collect user information and help them formulate relevant policies. This is a target optimization based on complete information, ignoring the protection of user privacy parameters. At the same time, in real life, the complete information of individuals cannot be obtained well, especially some privacy parameters. Therefore, when using traditional methods to formulate relevant policies, it is easy to cause information collection difficulties, which makes traditional methods unusable. Summary of the invention

[0005] In view of the problem that the energy trading based on edge computing in the prior art cannot achieve user privacy protection and utility balance, the present invention proposes an energy Internet trading method and system based on reinforcement learning blockchain empowerment. In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0006] An energy internet transaction method based on reinforcement learning blockchain empowerment includes the following steps:

[0007] S1, based on the energy trading relationship between operators, retailers and prosumers on the blockchain trading platform, a three-stage game model is constructed between operators, retailers and prosumers;

[0008] S2, using a distributed hierarchical policy gradient algorithm to solve the game equilibrium point in the three-stage game model, wherein the game equilibrium point includes an optimal unit service price, an optimal unit energy price, and an optimal energy demand;

[0009] S3, operators, retailers and producers and sellers conduct energy transactions based on the game equilibrium point obtained in step S2.

[0010] The step S2 comprises the following steps:

[0011] S2.1, set the network parameters of the three-stage game model;

[0012] S2.2, initializing the weight parameters of the three-stage game model;

[0013] S2.3, obtain the status of each operator Retailer status and the status of producers and sellers Each edge server in the blockchain trading platform uses the Markov decision process to convert the operator utility U o (η) is used as a reward function to select an appropriate unit service price η so that the operator utility U o (η) maximizes the retailer’s utility U r (p) is used as a reward function to select an appropriate unit energy price p so that the retailer's utility U r (p) maximize the utility of each producer and seller Selecting the appropriate energy demand q as a reward function j Producer-seller utility maximize.

[0014] In step S2.3, the operator's status The expression is:

[0015]

[0016] In the formula, p t-1represents the unit energy price at step t-1, represents the energy demand submitted by the prosumer to the local retailer through the edge server j at step t-1;

[0017] The operator utility U o The expression for maximizing (η) is:

[0018]

[0019] Where U m represents the additional reward obtained by the operator through the trusted blockchain service provided for each energy transaction, φ represents the transmission loss rate, c t represents the unit transmission cost, C o represents the fixed operation and maintenance cost, η min represents the minimum unit service price, η max represents the maximum unit service price, q j It represents the energy demand submitted by the prosumer to the local retailer through the edge server j, and N represents the set of all edge servers in the blockchain trading platform.

[0020] The additional reward U m The calculation formula is:

[0021] U m =(R f +rs)λ;

[0022] In the formula, R f represents the fixed block reward, r represents the blockchain service fee provided by the prosumer to the operator when each energy transaction occurs, s represents the block parameter, and λ represents the probability factor in the blockchain.

[0023] The status of the retailer The expression is:

[0024]

[0025] Where η t represents the unit service price at step t, represents the energy demand submitted by the prosumer to the local retailer through the edge server j at step t-1;

[0026] The retailer utility U r The expression for maximizing (p) is:

[0027]

[0028] In the formula, C g represents the production cost that retailers need to bear when producing energy, C srepresents the storage cost that retailers need to bear when storing energy, p min represents the minimum unit energy price, p max represents the maximum unit energy price, q j It represents the energy demand submitted by the prosumer to the local retailer through the edge server j, and N represents the set of all edge servers in the blockchain trading platform.

[0029] The production cost C g The calculation formula is:

[0030]

[0031] Where a, b, and k are weighting factors of the retailer’s power generation costs during production, and φ represents the transmission loss rate.

[0032] The storage cost C s The calculation formula is:

[0033]

[0034] In the formula, c s represents the unit cost of energy storage for retailers, ξ c represents the charging efficiency of the energy storage device, ξ d Represents the discharge efficiency of energy storage devices.

[0035] The status of the producer and seller The expression is:

[0036]

[0037] Prosumer Utility The expression for maximization is:

[0038]

[0039] In the formula, δ represents the conversion factor, w j represents the usage scenario of edge server j in terms of energy utilization, q min represents the minimum energy demand, q max represents the maximum energy demand, q j It represents the energy demand submitted by the prosumer to the local retailer through the edge server j, and r represents the blockchain service fee provided by the prosumer to the operator when each energy transaction occurs.

[0040] An energy Internet trading system based on reinforcement learning blockchain empowerment includes an energy application layer, an energy data layer and an edge control layer. The energy application layer interacts with the edge control layer through a smart contract interface, and the energy data layer interacts with the edge control layer; the energy application layer includes retailers and prosumers, and the retailers and prosumers interact through a blockchain trading platform; the edge control layer includes edge servers and distributed SDN controllers maintained by operators, and each edge server serves as a node in the blockchain trading platform; the energy data layer includes switches and energy routers, and the switches are connected to the distributed SDN controllers to receive scheduling instructions issued by the distributed SDN controllers and forward the scheduling instructions to the corresponding energy routers; the energy routers are used to sense the status of energy lines and reflect the status of energy lines to the edge servers.

[0041] The smart contract interface is established based on the smart contract system, which includes a user registration module, an energy trading module, an energy transmission module, an energy recording module and an information query module. After the three parties, namely, the operator, the producer and the retailer, register their respective accounts through the user registration module, the producer and the retailer place an order through the energy trading module according to their own needs, and the energy transaction is realized through the energy transmission module to realize the flow of energy from the retailer to the producer and the retailer; the energy recording module is used for retailers and producers and retailers to record their respective electricity information, and the information query module is used for all parties to query their own account information.

[0042] Beneficial effects of the present invention:

[0043] The hierarchical design of the multi-agent action and learning process in the present invention helps the agents learn their own strategies according to the competitive strategies and improve the overall performance of the trading system. Compared with the popular deep reinforcement learning algorithm, the present invention can help operators and retailers achieve higher utility, while producers and sellers can also achieve better utility. Under the unified pricing mechanism, the convergence order of different entities is consistent with the action order of the three stages of the Stackelberg game, which makes it more likely that the leader in the game will obtain better benefits than the follower. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 Schematic diagram of the transaction system of the present invention.

[0046] Figure 2 Schematic diagram of the three-stage Stackelberg game model.

[0047] Figure 3 This is the structural diagram of the hierarchical policy gradient algorithm.

[0048] Figure 4 This is a structural diagram of the smart contract system.

[0049] Figure 5 This is the game convergence performance under HDPG.

[0050] Figure 6 The performance comparison chart of different algorithms.

[0051] Figure 7 Schematic diagram of the utility under different parameters. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Embodiment 1: An energy internet transaction method based on reinforcement learning blockchain empowerment, comprising the following steps:

[0054] S1, based on the energy trading relationship between operators, retailers and prosumers on the blockchain trading platform, a three-stage game model is constructed between operators, retailers and prosumers;

[0055] like Figure 2 and Figure 3 As shown, the three-stage game model includes three strategy networks, which correspond to operators, retailers and producers and sellers respectively. Producers and sellers submit energy demands to operators through the blockchain trading platform. Operators, as intermediaries, assist in completing energy transactions and transmissions between retailers and producers and sellers through the blockchain trading platform. The blockchain trading platform is composed of several edge servers, each of which serves as a node in the blockchain trading platform and undertakes the functions of accounting, broadcasting, verification and consensus. The set of edge servers is represented by N = {1, 2, ..., j, ..., N'}.

[0056] S2, using the hierarchical distributed policy gradient algorithm (HDPG) to solve the game equilibrium point in the three-stage game model;

[0057] Considering the Stackelberg game under incomplete information, the Markov decision process is used to solve the game equilibrium point in the three-stage game model. A Markov decision process is equivalent to the entire process of energy transaction and transmission. The game equilibrium point includes the optimal unit service price, the optimal unit energy price and the optimal energy demand, so as to maximize the operator utility U o (η), retailer utility U r (p) and producer-seller utility Under the condition of maximizing the overall benefit, η represents the unit service price, p represents the unit energy price, q j represents the energy demand submitted by the prosumer to the local retailer through the edge server j, and j∈N.

[0058] The step S2 comprises the following steps:

[0059] S2.1, set the network parameters of the three policy networks;

[0060] The network parameters include the learning rate α of the retailer strategy network r , the learning rate α of the producer-seller strategy network p , the learning rate α of the operator strategy network o , discount factor γ.

[0061] S2.2, weight parameters for the three policy networks Perform random initialization;

[0062] S2.3, obtain the status of each operator Retailer status and the status of producers and sellers Each edge server uses a Markov decision process to convert the operator utility U o (η) is used as a reward function to select an appropriate unit service price η so that the operator utility U o (η) maximizes the retailer’s utility U r (p) is used as a reward function to select an appropriate unit energy price p so that the retailer's utility U r (p) maximize the utility of each producer and seller Choosing the right energy requirement as a reward function Producer-seller utility Maximize, and η t ∈A o , A o represents the action space of the operator, p t ∈A r , A r represents the retailer’s action space, A prepresents the action space of producers and marketers;

[0063] Status of the operator The expression is:

[0064]

[0065] In the formula, p t-1 represents the unit energy price at step t-1, represents the energy demand submitted by the prosumer to the local retailer through the edge server j at step t-1.

[0066] The operator utility U o The expression for maximizing (η) is:

[0067]

[0068] Where U m represents the additional reward obtained by the operator through the trusted blockchain service provided for each energy transaction, φ represents the transmission loss rate, c t represents the unit transmission cost, C o represents the fixed operation and maintenance cost, η min represents the minimum unit service cost, η max Indicates the maximum unit service cost.

[0069] The additional reward U m The calculation formula is:

[0070] U m =(R f +rs)λ;

[0071] In the formula, R f represents the fixed block reward, r represents the blockchain service fee provided by the producer and seller to the operator when each energy transaction occurs, s represents the block parameter, and λ represents the probability factor in the blockchain. m Operators can be further incentivized to maintain the blockchain.

[0072] The status of the retailer The expression is:

[0073]

[0074] The retailer utility U r (p) is determined by the unit energy price p and the total energy demand, and the retailer utility U r The expression for maximizing (p) is:

[0075]

[0076] In the formula, Cg represents the production cost that retailers need to bear when producing energy, C s represents the storage cost that retailers need to bear when storing energy, p min represents the minimum unit energy price, p max Represents the maximum unit energy price.

[0077] The production cost C g The calculation formula is:

[0078]

[0079] Where a, b, and k are weighting factors of power generation costs.

[0080] The storage cost C s The calculation formula is:

[0081]

[0082] In the formula, c s represents the unit cost of energy storage for retailers, ξ c represents the charging efficiency of the energy storage device, ξ d represents the discharge efficiency of the energy storage device, It represents the energy actually produced and stored by the retailer, taking into account the energy losses during transmission.

[0083] The status of the producer and seller The expression is:

[0084]

[0085] Prosumer Utility The expression for maximization is:

[0086]

[0087] In the formula, δ represents the conversion factor, w j represents the usage scenario of edge server j in terms of energy utilization, q min represents the minimum energy demand, q max represents the maximum energy demand, pq j represents the energy payment from the producer to the retailer, It represents the benefit obtained by edge server j in actual production by using the energy it purchased.

[0088] The specific process of the Markov decision process is a prior art, and the present invention will not repeat it. In addition, the outer loop in the present invention, that is, the total iteration of the Markov decision process, the middle loop, that is, the number of iterations of each strategy network, and the inner loop, that is, the iteration of the number of producers and sellers in the producer and seller strategy network, have a time complexity of O(E), O(T), and O(N) respectively. Each strategy network includes two fully connected layers, and the time complexity of each fully connected layer is expressed as Where K l refers to the number of fully connected neural units, and L represents the number of layers in a policy network. Because each policy network contains two fully connected layers to generate policies, the total time complexity of the algorithm is O(ETN(T f )).

[0089] Operator strategy network updates weight parameter θ o The formula is:

[0090]

[0091] In the formula, represents the operator's strategy at step t, represents the reward of the operator at step t, represents the state of the operator at step t, and S o represents the state space of the operator, α o Represents the learning rate of the operator strategy network.

[0092] The retailer strategy network updates the weight parameter θ r The formula is:

[0093]

[0094] In the formula, represents the retailer’s strategy at step t, represents the retailer’s reward at step t, represents the state of the retailer at step t, and S r represents the state space of the retailer, α r represents the learning rate of the retailer policy network.

[0095] Producer-seller strategy network updates weight parameters The formula is:

[0096]

[0097] In the formula, represents the strategy of the producer and marketer at step t, represents the reward of the producer and seller at step t, represents the state of the producer and seller at step t, and S p represents the state space of producers and sellers, represents the learning rate of the producer-seller strategy network.

[0098] In this embodiment, producers and sellers refer to distributed energy users who cannot produce energy by themselves or whose produced energy cannot meet their energy consumption. They can purchase energy from public energy companies or retailers of BSDEI according to their energy demand and unit energy price.

[0099] Retailers are energy users who use distributed generation equipment and energy storage equipment and whose total power generation is greater than their total energy consumption. Retailers earn revenue by providing energy to various distributed applications. On the other hand, they need to bear the costs of distributed generation and energy storage, and pay operators for transmission routing services.

[0100] The operator is the intermediary between producers and retailers, assisting in the energy trading process. In order to provide more convenient services and lower latency, the operator deploys hardware devices such as edge servers and distributed SDN controllers at the edge control layer to achieve edge-to-edge collaboration between various devices. In return, it charges retailers for transmission routing services and producers and retailers for trusted blockchain services.

[0101] Example 2: An energy internet trading system based on reinforcement learning blockchain empowerment, such as Figure 1As shown in the figure, the system includes an energy application layer, an energy data layer and an edge control layer, which together constitute an energy trading service system for a distributed energy market. The three layers are independent and interrelated, decoupling the energy routing and scheduling control in the blockchain-assisted energy internet (BEI); the energy application layer interacts with the edge control layer through a smart contract interface, and the energy data layer interacts with the edge control layer through a standard interface OpenFlow; the energy application layer includes retailers and producers and sellers, and retailers and producers and sellers directly exchange information through a blockchain trading platform. This direct transaction process can motivate retailers and producers and sellers to participate more actively in the distributed energy market. The blockchain trading platform provides a reliable and stable third-party service platform for energy transactions in the energy application layer; the edge control layer includes an edge server and a distributed SDN controller maintained by an operator, and the distributed SDN controller is used to schedule and control the energy routing of the energy data layer; each edge As a node in the blockchain trading platform, the edge server undertakes the functions of accounting, broadcasting, verification and consensus. The set of edge servers is represented by N = {1, 2, ..., j, ..., N'}; smart contracts provide reliable automated process control for energy transactions in the blockchain trading platform; the energy data layer includes switches and energy routers. The switches are used to receive scheduling instructions issued by the distributed SDN controller of the edge control layer and forward the scheduling instructions to the corresponding energy routers. The energy routers are used to sense the state of the energy line and reflect the real-time state of the energy line to the edge server, which helps the distributed SDN controller to modify the scheduling instructions; the state of the energy line includes the electric energy value, voltage and current on the energy line. In addition, the energy router can also receive commands from the distributed SDN controller to change the state of the energy router.

[0102] like Figure 4 As shown, the smart contract interface is established based on the smart contract system, and the smart contract system includes a user registration module, an energy trading module, an energy transmission module, an energy recording module and an information query module. After the three parties, the operator, the producer and the retailer, register their respective accounts through the user registration module, the producer and the retailer place an order through the energy trading module according to their own needs, and the energy transaction realizes the flow of energy from the retailer to the producer and the retailer through the energy transmission module; the energy recording module is used for retailers and producers and retailers to record their respective electricity information, and the information query module is used for all parties to query their own account information.

[0103] Specifically, in the user registration module, the smart contract deployer is the initial administrator of the trading system and initializes some public parameters. Participants need to register an account through the user registration module according to their user name, account address and account type. Then, the information of these accounts is initialized, including available energy and energy coins, as well as the total energy generated and used. Considering the cold start problem, participants can obtain energy coins through the energy trading module. After that, the producer and seller confirms the exact amount of energy required and places an order, and then completes the transaction process through the energy trading module, and the entire energy trading process is traded based on the method described in Example 1. The energy trading module can verify the authority of the account, which also ensures the sufficient balance of energy; then the energy transmission module is called, which not only clears the transaction between the retailer and the operator, but also activates the energy scheduling switch to ensure that energy flows from the retailer to the producer and seller; in the energy recording module, according to the smart meter data at the location of the retailer and the producer and seller, the total power generation and total power usage can be modified through the energy recording module, and then the cost is calculated according to the power; the information query module provides six types of interfaces for all participants to query their account information.

[0104] As follows, one operator, one retailer, and 10 edge servers are set up to illustrate the performance of the present invention from the perspective of convergence performance under a unified pricing mechanism. Since the optimal requirements of edge servers are very similar under a unified pricing mechanism, such as Figure 5 As shown in the figure, by selecting one of the edge servers for display, both the operator and the retailer obtain quite good utility. Under the high-level strategy of the operator and the middle-level strategy of the retailer, the prosumer, that is, the consumer, can also quickly converge to a relatively good solution. The convergence order of different entities is consistent with the action order of the three stages of the Stackelberg game, so the leader is more likely to obtain better benefits than the follower.

[0105] In order to demonstrate the excellent performance of the present invention, Figure 6 As shown, the present invention is compared with some popular deep reinforcement learning algorithms from the perspective of economic analysis. In order to reduce random errors, the data in the figure are the average of 10 experimental results. Figure 6 It can be clearly seen that HDPG obtains more total rewards than the three deep reinforcement learning algorithms, namely PPO also known as neighboring policy optimization, SAC also known as flexible actor-critic, and DQN also known as deep Q learning. In addition, different algorithms have their own characteristics. For example, SAC assists retailers in obtaining the highest utility, but performs poorly in the operator's strategy. In contrast, HDPG helps operators and retailers achieve higher utility. In addition, edge devices also achieve good utility using HDPG. The hierarchical design of multi-agent actions and learning processes helps agents learn their own strategies based on competitive strategies, which is a potential reason for the better performance of HDPG.

[0106] like Figure 7 As shown in Figure 2, the parameter sensitivity of the utility is analyzed as the number of production users changes. Since the utility of the edge server is not very sensitive to the number of participants, a box plot is used in the figure to carefully describe the impact of the edge server energy usage on the edge server utility. Figure 7 As shown in a, it is sensible for edge servers with high-value production to participate in the energy market. Figure 7 b and Figure 7 c It can be seen that the transmission loss rate largely determines the utility of operators and retailers. Therefore, it is of great significance to adopt more advanced technologies in energy transmission and distribution networks to reduce the transmission loss rate. As the number of edge servers increases, the utility of operators and retailers increases steadily, while the utility of each edge server shows a slight downward trend. It should be pointed out that the increase in the number of edge servers may affect the strategy of each edge server, resulting in certain fluctuations in the utility of all entities. Figure 7 d shows the changing trend of retailers’ utility under different energy storage efficiencies. Low energy storage efficiency may lead to negative utility, while the higher the efficiency of energy storage equipment, the better the retailer’s utility. However, the cost and difficulty of reducing transmission loss rate and improving energy storage efficiency are also difficulties in the energy trading market.

[0107] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An energy internet transaction method based on reinforcement learning blockchain empowerment, characterized in that: The steps include: S1, based on the energy trading relationship between operators, retailers and prosumers on the blockchain trading platform, a three-stage game model is constructed between operators, retailers and prosumers; S2, using a distributed hierarchical policy gradient algorithm to solve the game equilibrium point in the three-stage game model, wherein the game equilibrium point includes an optimal unit service price, an optimal unit energy price, and an optimal energy demand; S3, operators, retailers and producers and sellers conduct energy transactions based on the game equilibrium point obtained in step S2; The blockchain trading platform includes a number of interconnected edge servers, each of which is connected to a distributed SDN controller through a smart contract interface. Retailers and producers and sellers realize energy transactions through edge servers and smart contract interfaces, and producers and sellers provide blockchain services to retailers and producers and sellers through edge servers and smart contract interfaces; the distributed SDN controller is connected to switches and energy routers, and the distributed SDN controller, switches and energy routers are used to realize the scheduling and transmission of transaction energy; The step S2 comprises the following steps: S2.1, set the network parameters of the three-stage game model; S2.2, initializing the weight parameters of the three-stage game model; S2.3, obtain the status of each operator Retailer status and the status of producers and sellers Each edge server in the blockchain trading platform uses the Markov decision process to convert the operator utility U o (η) is used as a reward function to select an appropriate unit service price η so that the operator utility U o (η) maximizes the retailer's utility U r (p) is used as a reward function to select an appropriate unit energy price p so that the retailer's utility U r (p) maximize the utility of each producer and seller Selecting the appropriate energy demand q as a reward function j Producer-seller utility maximize; The operator utility U o The expression for maximizing (η) is: Where U m represents the additional reward obtained by the operator through the trusted blockchain service provided for each energy transaction, φ represents the transmission loss rate, c t represents the unit transmission cost, C o represents the fixed operation and maintenance cost, η min represents the minimum unit service price, η max represents the maximum unit service price, q j represents the energy demand submitted by the prosumer to the local retailer through the edge server j, and N represents the set of all edge servers in the blockchain trading platform; The additional reward U m The calculation formula is: U m =(R f +rs)λ; In the formula, R f represents the fixed block reward, r represents the blockchain service fee provided by the prosumer to the operator when each energy transaction occurs, s represents the block parameter, and λ represents the probability factor in the blockchain; The retailer utility U r The expression for maximizing (p) is: In the formula, C g represents the production cost that retailers need to bear when producing energy, C s represents the storage cost that retailers need to bear when storing energy, p min represents the minimum unit energy price, p max represents the maximum unit energy price; Prosumer Utility The expression for maximization is: In the formula, δ represents the conversion factor, w j represents the usage scenario of edge server j in terms of energy utilization, q min represents the minimum energy demand, q max represents the maximum energy demand; The production cost C g The calculation formula is: In the formula, a, b, k are weighting factors of the retailer’s power generation cost during production, and φ represents the transmission loss rate; The storage cost C s The calculation formula is: In the formula, c s represents the unit cost of energy storage for retailers, ξ c represents the charging efficiency of the energy storage device, ξ d Represents the discharge efficiency of energy storage devices.

2. The energy internet transaction method based on reinforcement learning blockchain empowerment according to claim 1 is characterized in that: In step S2.3, the operator's status The expression is: In the formula, p t-1 represents the unit energy price at step t-1, represents the energy demand submitted by the prosumer to the local retailer through the edge server j at step t-1.

3. The energy internet transaction method based on reinforcement learning blockchain empowerment according to claim 1 is characterized in that: The status of the retailer The expression is: Where η t represents the unit service price at step t, represents the energy demand submitted by the prosumer to the local retailer through the edge server j at step t-1.

4. The energy internet transaction method based on reinforcement learning blockchain empowerment according to claim 1 is characterized in that: The status of the producer and seller The expression is: In the formula, p t represents the unit energy price at step t.

Images