Distributed new energy settlement and control system based on block chain

Abstract

The invention relates to the technical field of new energy operation, and discloses a distributed new energy settlement and control system based on a block chain, and the system collects new energy power generation, energy storage and user power utilization data through an Internet of Things collector, and builds a network model comprising new energy production, energy storage, consumption and power grid nodes. And a Markov decision process is adopted for modeling. The system further defines a state space, an action space and a reward function, and a deep reinforcement learning algorithm optimization strategy is applied to maximize long-term rewards. Besides, a block chain technology is introduced, transparent and safe recording of new energy production, transaction, consumption and settlement processes is realized, transaction settlement is automatically executed through an intelligent contract, and the efficiency and reliability of energy transaction are improved.

Description

Technical Field

[0001] The present invention relates to the field of new energy operation technology, and specifically to a distributed new energy settlement and control system based on blockchain. Background Art

[0002] With the transformation of the global energy structure and the popularization of the concept of sustainable development, distributed new energy systems have gradually become a research hotspot in the energy field. Distributed new energy systems usually include new energy power generation equipment (such as solar photovoltaic panels, wind turbines, etc.), energy storage equipment (such as battery energy storage systems, pumped storage power stations, etc.), user power equipment and grid nodes, which together constitute a complex and huge energy network. However, in this network, the generation, storage, transmission and consumption of new energy face many challenges, such as mismatch between energy supply and demand, opaque energy trading, poor grid stability, etc. These problems seriously restrict the efficient operation and widespread application of distributed new energy systems. Summary of the invention

[0003] The purpose of the present invention is to provide a distributed new energy settlement and control system based on blockchain to solve the problems raised in the above background technology.

[0004] To achieve the above purpose, the present invention provides the following technical solution: a distributed new energy settlement and control system based on blockchain, the system comprising: Energy data collection module: used to collect energy data of new energy power generation equipment, energy storage equipment and user electrical equipment through the IoT collector; New energy network management module: Based on the data collected by the energy data collection module, using blockchain technology, a distributed new energy network model including new energy production nodes, energy storage nodes, consumption nodes and grid nodes is constructed and maintained. The model models the generation, storage, transmission and consumption process of new energy as a Markov decision process; The new energy network management module further includes: State space definition submodule: used to define the state space of the system, including the capacity dimension of the new energy production node, the energy storage state dimension of the energy storage node, the demand dimension of the consumer node, and the transmission state dimension of the power grid node; Action space definition submodule: used to define the possible action space of the system, including adjusting the operating parameters of new energy production equipment, controlling the charging and discharging strategy of energy storage equipment, optimizing the dispatching plan of the power grid, and starting or shutting down backup energy equipment; Reward function design submodule: Based on the utilization rate of new energy, energy trading efficiency, grid stability, operation and maintenance costs, and environmental benefits, the reward function is defined to evaluate the actions taken by the system and give corresponding rewards or penalties; Strategy optimization submodule: uses deep reinforcement learning algorithms to continuously optimize strategies based on historical data and real-time feedback to maximize long-term cumulative rewards; Blockchain settlement and recording submodule: records the production, transaction, consumption and settlement process of new energy; automatically executes energy transactions and settlements through smart contracts.

[0005] Preferably, the production capacity dimension of the new energy production node includes power generation and power generation efficiency; the energy storage status dimension of the energy storage node includes power amount and charging and discharging rate; the demand dimension of the consumer node includes power consumption, power consumption period and power consumption type; the transmission status dimension of the power grid node includes voltage, current, power factor and line loss.

[0006] Preferably, for the new energy utilization rate in the reward function, the reward value is calculated using the following formula: ,in, represents the total production of new energy, Indicates the amount of energy wasted due to various reasons, is the weight coefficient of new energy utilization rate.

[0007] Preferably, for the energy transaction efficiency in the reward function, the reward value is calculated using the following formula: ,in, Represents the energy supply, represents the energy demand, is the weight coefficient of energy trading efficiency.

[0008] Preferably, for the energy transaction efficiency in the reward function, the reward value is calculated using the following formula: ,in, represents the actual value of the ith parameter of the power grid, Indicates the reference value of the parameter. Indicates the maximum allowed value of the parameter. is the number of parameters, is the weight coefficient of power grid stability.

[0009] Preferably, the training process of the distributed new energy network model includes a model initialization phase and a model training phase.

[0010] Preferably, the model initialization stage includes: Set the initial state space S of the system, including the initial capacity of the new energy production node, the initial energy storage state of the energy storage node, the initial demand of the consumption node and the initial transmission state of the power grid node; Set the action space A, including all possible actions, including adjusting the operating parameters of new energy production equipment, controlling the charging and discharging strategies of energy storage equipment, optimizing the dispatching plan of the power grid, and starting or shutting down backup energy equipment; Initialize the neural networks in the deep reinforcement learning algorithm, including the actor network and the critic network, as well as the model's hyperparameters, including the learning rate and discount factor; Initialize the blockchain settlement and recording submodule to ensure the correct deployment and initialization of smart contracts.

[0011] Preferably, the model training stage includes: For each time step t: Step 1: Observe the current state ; Step 2: Select actions based on the current policy output by the actor network ; Step 3: Perform the action , and observe the next state and instant rewards ; Step 4: Transfer the state tuple Stored in the experience replay buffer; Step 5: Randomly sample a batch of experience tuples from the experience replay buffer to update the actor network and the critic network; Step 6: Use the critic network to estimate the state value function or action value function and calculate the advantage function; Step 7: Update the actor network using the advantage function; Step 8: Update the critic network using the advantage function; Repeat steps 1-8 until the preset number of training rounds is reached.

[0012] Preferably, the blockchain settlement and recording submodule designs smart contracts according to the business logic of distributed new energy settlement and control, including the production, trading, consumption and settlement rules of new energy.

[0013] Preferably, the data structure defined in the smart contract includes user accounts, energy transaction records and settlement status; and the smart contract contains functions for triggering settlement and recording operations.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The energy data acquisition module collects energy data from new energy power generation equipment, energy storage equipment and user power equipment in real time, providing an accurate data basis for the new energy network management module. Combined with the Markov decision process model, the system can dynamically predict and adjust the generation, storage, transmission and consumption of new energy, effectively alleviate the problem of energy supply and demand mismatch, and improve energy utilization efficiency and the overall operation efficiency of the system.

[0015] The blockchain settlement and recording submodule uses the decentralized, tamper-proof and traceable characteristics of blockchain technology to record the entire process of new energy production, trading, consumption and settlement. All transaction information is open and transparent, and is automatically executed through smart contracts, ensuring the fairness, security and efficiency of energy transactions and reducing transaction costs and risks.

[0016] The new energy network management module defines the state space and action space of the system, and designs a strategy optimization sub-module based on multi-dimensional reward functions such as new energy utilization rate, energy trading efficiency, and grid stability. It can use deep reinforcement learning algorithms to continuously optimize the grid's scheduling plan and the charging and discharging strategy of energy storage equipment. This not only improves the grid's adaptability and flexibility to new energy access, but also significantly enhances the grid's stability and reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is the overall structure diagram of the distributed new energy settlement and control system based on blockchain described in the present invention; Figure 2 A flow chart for building a distributed new energy network model; Figure 3 Schematic diagram of the reward function design. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] See also Figure 1-3 The present invention provides a technical solution: a distributed new energy settlement and control system based on blockchain, the system comprising: Energy data collection module: The energy data collection module of this system collects energy data of new energy power generation equipment (such as solar photovoltaic panels, wind turbines, etc.), energy storage equipment (such as battery energy storage systems, pumped storage power stations, etc.) and user electrical equipment in real time through the Internet of Things collector. The energy data includes but is not limited to power generation, energy storage status, power consumption, equipment operation status, etc., providing basic data support for subsequent new energy network management.

[0020] The new energy network management module builds and maintains a distributed new energy network model including new energy production nodes, energy storage nodes, consumption nodes and grid nodes based on the data provided by the energy data acquisition module. The model models the generation, storage, transmission and consumption process of new energy as a Markov decision process for dynamic decision making and optimization. It further includes: The state space definition submodule is responsible for defining the state space of the system, including the capacity dimension of the new energy production node (such as current power generation, power generation efficiency, etc.), the energy storage state dimension of the energy storage node (such as the remaining power of the energy storage device, the charging and discharging rate, etc.), the demand dimension of the consumer node (such as user power consumption, power consumption period and power consumption type, etc.) and the transmission state dimension of the power grid node (such as power grid voltage, current, power factor and line loss, etc.).

[0021] Action space definition submodule, which defines the possible action space of the system, including adjusting the operating parameters of new energy production equipment (such as adjusting the tilt angle of photovoltaic panels, the speed of wind turbines, etc.), controlling the charging and discharging strategies of energy storage equipment (such as deciding when to charge, when to discharge, and the rate of charging and discharging, etc.), optimizing the scheduling plan of the power grid (such as adjusting the power distribution of each node, starting or shutting down certain power lines, etc.) and starting or shutting down backup energy equipment (such as backup generators, energy storage equipment, etc.).

[0022] Reward function design submodule, the reward function design submodule designs a comprehensive reward function based on multiple dimensions such as new energy utilization rate, energy trading efficiency, power grid stability, operation and maintenance costs and environmental benefits. This function evaluates the actions taken by the system and gives corresponding rewards or penalties. For example, when the system improves the utilization rate of new energy and energy trading efficiency through optimized scheduling, it will receive positive rewards; when the system causes the stability of the power grid to decline or the operation and maintenance costs to increase, it will be punished.

[0023] The strategy optimization submodule uses a deep reinforcement learning algorithm to continuously optimize the system's decision-making strategy based on historical data and real-time feedback. The algorithm gradually finds a strategy that can maximize long-term cumulative rewards through continuous trial and error and learning. During the training process, the system selects actions based on the current state, observes the results after execution and obtains rewards or penalties, and then adjusts the strategy based on this information to improve future decisions.

[0024] Blockchain settlement and recording submodule, which uses blockchain technology to record the production, transaction, consumption and settlement process of new energy. The distributed ledger characteristics of blockchain ensure the transparency, security and immutability of data. Through the automatic execution of energy transactions and settlements by smart contracts, the system can achieve efficient and accurate energy transaction management. Smart contracts are self-executing contracts. When specific conditions are met, it will automatically perform preset operations, such as transfers, recording transaction information, etc., which greatly reduces transaction costs and transaction time, and improves the efficiency and reliability of energy transactions.

[0025] The present invention will be further described below in conjunction with Examples 1 to 3: Embodiment 1:

[0026] The design of the reward function includes the utilization rate of new energy (weight coefficient α), energy trading efficiency (weight coefficient β), grid stability (weight coefficient γ), operation and maintenance cost (weight coefficient δ, note that it is used as a penalty term here) and environmental benefit (weight coefficient ϵ). These weight coefficients can be adjusted according to the needs and preferences of the actual system.

[0027] New energy utilization rate is an important indicator to measure the effective use of new energy by the system. It can be calculated by the following formula: ,in, is the total production of renewable energy, such as electricity from solar panels or wind turbines; It is the amount of energy wasted due to equipment failure, insufficient energy storage, or improper scheduling. For example, if the solar panels generate a total of 1000 kilowatt-hours (kWh) of electricity in a day, but 100 kWh of electricity is wasted due to limited storage capacity or improper grid scheduling, then the utilization rate of new energy is 90%, and the corresponding reward value (assuming α = 0.5) is 0.5×0.9=0.45.

[0028] Energy trading efficiency reflects the matching degree between supply and demand. It can be calculated by the following formula: ,in, is the energy supply, is the energy demand. For example, if the power supplied in a certain period is 500 kWh and the demand is 450 kWh, then the energy trading efficiency is 90% (because the difference between supply and demand is small), and the corresponding reward value (assuming β = 0.3) is 0.3 × 0.9 = 0.27.

[0029] Grid stability is key to ensuring the continuity and reliability of electricity supply. It can be calculated by monitoring the fluctuation of parameters such as voltage, frequency and power factor of the power grid. The specific formula is as follows: ,in, is the actual value of the ith parameter of the power grid, is the reference value of this parameter, is the maximum allowed value of this parameter, is the number of parameters. For example, if the actual value of the voltage is 220V, the reference value is 230V, the maximum allowed value is 240V, and this is the only parameter monitored in the grid (n=1), then the stability bonus (assuming γ=0.1) will be calculated based on how much the voltage deviates from the reference value.

[0030] The operation and maintenance cost is an inevitable part of the system operation, which is deducted from the total reward as a penalty item. It can be calculated by the following formula: ,in, is the operation and maintenance cost, including equipment maintenance, staff wages, fault repair, etc. For example, if the operation and maintenance cost in one day is 1,000 yuan, and the weight coefficient of the operation and maintenance cost is δ=0.05, then the penalty value is -0.05×1000=-50.

[0031] Environmental benefits take into account the positive impact of the system on the environment, such as reducing carbon emissions. It can be calculated by the following formula: ,in is the amount of energy saved by optimization, is the amount of carbon emissions reduced per unit of energy saved. For example, if 100 kWh of electricity is saved through optimized scheduling, and every kWh of electricity saved can reduce 0.5 kg of carbon emissions, then the environmental benefit reward (assuming ϵ = 0.1) is 0.1 × 100 × 0.5 = 5.

[0032] Finally, add up the reward values ​​(or penalty values) of the above factors to get the total reward value R: , this total reward value is used to evaluate the actions taken by the system, and the strategy is continuously optimized through a deep reinforcement learning algorithm to maximize the long-term cumulative reward. Embodiment 2:

[0033] The training process of the distributed new energy network model includes a model initialization phase and a model training phase. The following further describes the training process of the distributed new energy network model in conjunction with specific embodiments.

[0034] ① Initialization phase: Set the initial state space S of the system. Taking a distributed energy system demonstration area as an example, the state space S includes: The initial capacity dimension of new energy production nodes (such as solar photovoltaic panels, wind turbines, etc.) assumes that there are 15 photovoltaic units with a capacity of 0.1 MW each, totaling 1.5 MW; and 5 wind turbines with a capacity of 1.5 MW each, totaling 7.5 MW.

[0035] The initial energy storage state dimension of the energy storage node (such as battery energy storage, supercapacitor, etc.) assumes that the electrical energy storage capacity in the demonstration area is 2 MW·h.

[0036] The initial demand dimension of the consumption node is the initial electricity demand of users in the demonstration area.

[0037] The initial transmission state dimensions of the grid nodes include parameters such as voltage, frequency, and power factor.

[0038] Set the action space A. The action space A includes: Adjust the operating parameters of new energy production equipment, such as the angle of photovoltaic panels, the speed of wind turbines, etc.

[0039] Control the charging and discharging strategies of energy storage devices, such as setting charging power, discharging power, etc.

[0040] Optimize the dispatching plan of the power grid, such as adjusting the transmission power and frequency of the power grid nodes.

[0041] Start or shut down backup energy equipment such as gas turbines, fuel cells, etc.

[0042] Initialize the neural network in the deep reinforcement learning algorithm. Take the DDPG algorithm as an example to initialize the actor network and the critic network, and set hyperparameters such as learning rate and discount factor.

[0043] Initialize the blockchain settlement and recording submodules, deploy smart contracts, and ensure the normal operation of the blockchain network.

[0044] ② Model training phase For each time step t: Step 1: Observe the current state , such as the current photovoltaic power generation, wind turbine power generation, energy storage, user electricity demand, etc.

[0045] Step 2: Select an action based on the current policy (output by the actor network) , such as adjusting the angle of photovoltaic panels, setting the charging power of energy storage devices, etc.

[0046] Step 3: Perform the action , and observe the next state and instant rewards . Instant Rewards It can be calculated based on the reward function design submodule, taking into account factors such as new energy utilization, energy trading efficiency, grid stability, operation and maintenance costs, and environmental benefits.

[0047] Step 4: Transfer the state tuple Stored into the experience replay buffer.

[0048] Step 5: Randomly sample a batch of experience tuples from the experience replay buffer to update the actor network and the critic network.

[0049] Step 6: Use the critic network to estimate the state value function or action value function and calculate the advantage function.

[0050] Step 7: Update the actor network using the advantage function to optimize the policy.

[0051] Step 8: Update the critic network using to more accurately estimate the state value or action value.

[0052] Repeat steps 1-8 until the preset number of training rounds is reached or other stopping conditions are met.

[0053] The advantage function mentioned above uses advantage estimation based on the action value function. During the training process, the performance indicators of the strategy are constantly monitored, such as long-term cumulative rewards, new energy utilization rate, etc. When the performance indicators of the strategy reach the preset target or no longer improve significantly, the strategy is considered to have converged. Save the trained neural network model as the optimal strategy of the system. Embodiment 3:

[0054] The blockchain settlement and recording submodule is responsible for designing and executing smart contracts based on the business logic of distributed new energy to ensure the security and transparency of the production, trading, consumption and settlement of new energy. The following details the design of smart contracts, their data structure and functional implementation.

[0055] Smart contract design: Smart contracts are automated scripts on the blockchain that are automatically executed when specific conditions are met. In this system, smart contracts mainly include the following key parts: data structure definition, business logic rules, and trigger functions.

[0056] ① Data structure definition: User Account: Data structure: UserAccount; The fields include: accountID: user unique identifier; balance: account balance, which indicates the amount of energy coins or currency currently held by the user; energyProduced: the total amount of new energy produced by the user; energyConsumed: the total amount of new energy consumed by the user; Energy transaction records: Data structure: TransactionRecord; The fields include: transactionID: transaction unique identifier; sellerID: seller user ID; buyerID: buyer user ID; energyAmount: transaction energy amount; price: transaction price; timestamp: transaction timestamp; Settlement status: Data structure: SettlementStatus; The fields include: transactionID: the associated transaction ID; status: settlement status (such as "pending settlement", "settled", "settlement failed"); settlementTime: settlement timestamp (if settled); ② Business logic rules: The rules for new energy production, trading, consumption and settlement are encoded in the smart contract: Renewable energy production: When a user’s renewable energy equipment (such as solar panels) generates electricity, the system uploads the production data to the blockchain through the Internet of Things (IoT) device, and the smart contract automatically updates the user’s energyProduced field.

[0057] Energy transaction: After the buyer and seller reach an agreement, the smart contract creates a new TransactionRecord to record the transaction details, deducts the corresponding amount of energy from the seller's account, adds it to the energyConsumed of the buyer's account, and updates the balance of both parties.

[0058] Energy consumption: When a user consumes new energy, the smart contract updates the user’s energyConsumed field based on the consumption data and deducts the corresponding fee from his account balance.

[0059] Settlement rules: The smart contract regularly checks transaction records. For transactions that meet the settlement conditions (such as confirmation by both parties, reaching the settlement period, etc.), it automatically executes the settlement logic, updates the SettlementStatus, and records the settlement results on the blockchain.

[0060] ③ Trigger function: Trigger settlement function: triggerSettlement(transactionID); Input parameters: transactionID (ID of the transaction to be settled); Function: Check whether the transaction meets the settlement conditions. If so, update the transaction status to "settled" and record the settlement time.

[0061] Record operation function: recordTransaction(sellerID, buyerID, energyAmount, price); Input parameters: sellerID (seller ID), buyerID (buyer ID), energyAmount (transaction energy amount), price (transaction price); Function: Creates a new transaction record, updates the seller's and buyer's account balances and energy amounts, and triggers the settlement process (if applicable).

[0062] Assume that user A (accountID: 0x1) and user B (accountID: 0x2) conduct a new energy transaction, where A sells 100 kWh of electricity to B at a price of 0.5 energy coins per kWh. The transaction process is as follows: Transaction initiation: After A and B reach an agreement outside the blockchain, the system calls recordTransaction(0x1, 0x2,100, 0.5).

[0063] Transaction record: The smart contract executes the recordTransaction function, creates a new TransactionRecord, updates A’s energyProduced and B’s energyConsumed fields, and deducts 50 energy coins from A’s account balance and adds them to B’s account balance.

[0064] Settlement trigger: Assuming that the system sets the transaction to be settled immediately, the smart contract automatically calls the triggerSettlement(transactionID) function, checks the transaction record, and after confirming that the transaction is valid, updates the SettlementStatus of the transaction to "settled" and records the settlement time.

[0065] Through the above design, smart contracts ensure data integrity, transparency and automation in the settlement and control process of distributed new energy, reduce transaction costs and improve settlement efficiency.

[0066] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0067] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A distributed new energy settlement and control system based on blockchain, characterized in that: The system comprises: Energy data collection module: used to collect energy data of new energy power generation equipment, energy storage equipment and user electrical equipment through the IoT collector; New energy network management module: Based on the data collected by the energy data collection module, using blockchain technology, a distributed new energy network model including new energy production nodes, energy storage nodes, consumption nodes and grid nodes is constructed and maintained. The model models the generation, storage, transmission and consumption process of new energy as a Markov decision process; The new energy network management module further includes: State space definition submodule: used to define the state space of the system, including the capacity dimension of the new energy production node, the energy storage state dimension of the energy storage node, the demand dimension of the consumer node, and the transmission state dimension of the power grid node; Action space definition submodule: used to define the possible action space of the system, including adjusting the operating parameters of new energy production equipment, controlling the charging and discharging strategy of energy storage equipment, optimizing the dispatching plan of the power grid, and starting or shutting down backup energy equipment; Reward function design submodule: Based on the utilization rate of new energy, energy trading efficiency, grid stability, operation and maintenance costs, and environmental benefits, the reward function is defined to evaluate the actions taken by the system and give corresponding rewards or penalties; Strategy optimization submodule: uses deep reinforcement learning algorithms to continuously optimize strategies based on historical data and real-time feedback to maximize long-term cumulative rewards; Blockchain settlement and recording submodule: records the production, transaction, consumption and settlement process of new energy; automatically executes energy transactions and settlements through smart contracts.

2. The distributed new energy settlement and control system based on blockchain according to claim 1 is characterized in that: The capacity dimension of the new energy production node includes power generation and power generation efficiency; the energy storage status dimension of the energy storage node includes power and charging and discharging rate; the demand dimension of the consumer node includes power consumption, power consumption period and power consumption type; the transmission status dimension of the power grid node includes voltage, current, power factor and line loss.

3. The distributed new energy settlement and control system based on blockchain according to claim 2 is characterized in that: For the new energy utilization rate in the reward function, the reward value is calculated using the following formula: ,in, represents the total production of new energy, Indicates the amount of energy wasted due to various reasons, is the weight coefficient of new energy utilization rate.

4. The distributed new energy settlement and control system based on blockchain according to claim 2 is characterized in that: For the energy transaction efficiency in the reward function, the reward value is calculated using the following formula: ,in, represents the energy supply, represents the energy demand, is the weight coefficient of energy trading efficiency.

5. The distributed new energy settlement and control system based on blockchain according to claim 2 is characterized in that: For the energy transaction efficiency in the reward function, the reward value is calculated using the following formula: ,in, represents the actual value of the ith parameter of the power grid, Indicates the reference value of the parameter. Indicates the maximum allowed value of the parameter. is the number of parameters, is the weight coefficient of power grid stability.

6. The distributed new energy settlement and control system based on blockchain according to claim 2 is characterized in that: The training process of the distributed new energy network model includes a model initialization phase and a model training phase.

7. The distributed new energy settlement and control system based on blockchain according to claim 6 is characterized in that: The model initialization phase includes: Set the initial state space S of the system, including the initial capacity of the new energy production node, the initial energy storage state of the energy storage node, the initial demand of the consumption node and the initial transmission state of the grid node; Set the action space A, including all possible actions, including adjusting the operating parameters of new energy production equipment, controlling the charging and discharging strategies of energy storage equipment, optimizing the dispatching plan of the power grid, and starting or shutting down backup energy equipment; Initialize the neural networks in the deep reinforcement learning algorithm, including the actor network and the critic network, as well as the model's hyperparameters, including the learning rate and discount factor; Initialize the blockchain settlement and recording submodule to ensure the correct deployment and initialization of smart contracts.

8. The distributed new energy settlement and control system based on blockchain according to claim 4 is characterized in that: The model training phase includes: For each time step t: Step 1: Observe the current state ; Step 2: Select actions based on the current policy output by the actor network ; Step 3: Perform the action , and observe the next state and instant rewards ; Step 4: Transfer the state tuple Stored in the experience replay buffer; Step 5: Randomly sample a batch of experience tuples from the experience replay buffer to update the actor network and the critic network; Step 6: Use the critic network to estimate the state value function or action value function and calculate the advantage function; Step 7: Update the actor network using the advantage function; Step 8: Update the critic network using the advantage function; Repeat steps 1-8 until the preset number of training rounds is reached.

9. The distributed new energy settlement and control system based on blockchain according to claim 2 is characterized in that: The blockchain settlement and recording submodule designs smart contracts based on the business logic of distributed new energy settlement and control, including the production, trading, consumption and settlement rules of new energy.

10. The distributed new energy settlement and control system based on blockchain according to claim 9 is characterized in that: The data structure defined in the smart contract includes user accounts, energy transaction records and settlement status; and the smart contract contains functions for triggering settlement and recording operations.

Images