Distributed power supply transaction intelligent clearing and optimizing method and system based on deep reinforcement learning

Abstract

The invention discloses a distributed power supply transaction intelligent clearing and optimizing method and system based on deep reinforcement learning, and relates to the technical field of distributed power supply transaction and artificial intelligence. The method comprises the following steps: collecting distributed power supply transaction full-scene multi-dimensional data, and preprocessing the data to generate a state vector representing a market state; inputting the state vector into a hybrid intelligent optimization model fusing a learning type strategy gradient and an improved white head eagle search algorithm, and obtaining and outputting an optimal clearing strategy; based on the optimal clearing strategy, automatically executing a clearing process through an intelligent contract deployed on a block chain, and synchronously storing the clearing process and result data as historical clearing data in the block chain; and calling the historical clearing data from a block chain, and performing feedback optimization on core parameters of the hybrid intelligent optimization model according to the historical clearing data. Therefore, autonomous optimization and credible execution of the distributed power supply transaction clearing strategy are realized.

Description

Technical Field

[0001] The embodiments of the present invention relate to the fields of distributed power trading technology and artificial intelligence technology, and particularly to a method and system for intelligent clearing and optimization of distributed power trading based on deep reinforcement learning. Background Technology

[0002] With the deepening of power system reform, distributed power sources (such as photovoltaic, wind power, and small hydropower) have developed rapidly due to their clean and flexible characteristics, gradually becoming an important part of the energy supply system. Distributed power trading is characterized by "multiple participants, high frequency, complex data, and strong constraints." Traditional centralized trading models face many bottlenecks: on the one hand, traditional reinforcement learning algorithms that rely on manually designed value functions and time-series difference rules are difficult to adapt to dynamically changing market environments and complex network constraints, resulting in insufficient flexibility in clearing strategies; on the other hand, univariate heuristic optimization algorithms are prone to getting trapped in local optima and cannot achieve global optimization of clearing strategies; at the same time, the lack of trust among participants in distributed trading, the risk of data tampering, and the lack of transparency in the trading process further restrict trading efficiency and fairness.

[0003] While existing blockchain technology offers decentralized, immutable, and traceable reliability, it lacks deep integration with intelligent optimization algorithms, hindering the intelligent and adaptive clearing of transactions. Traditional clearing algorithms often rely on fixed rules and cannot autonomously learn optimal strategies to balance multiple optimization objectives such as transaction efficiency, price fairness, and grid losses. Therefore, there is an urgent need to construct a closed-loop system integrating advanced intelligent optimization algorithms and blockchain technology to address core issues in distributed power generation transactions, including insufficient strategy learning, weak global optimization capabilities, and inadequate transaction reliability. Summary of the Invention

[0004] This invention addresses the technical pain points of distributed power source trading by combining the advantages of deep reinforcement learning, metaheuristic optimization, and blockchain technology. It provides a method and system for intelligent clearing and optimization of distributed power source trading based on deep reinforcement learning, achieving autonomous optimization and reliable execution of the clearing strategy for distributed power source trading.

[0005] In a first aspect, embodiments of the present invention provide a method for intelligent clearing and optimization of distributed power source transactions based on deep reinforcement learning, comprising:

[0006] Collect multi-dimensional data from the entire scenario of distributed power trading, and preprocess the multi-dimensional data to generate a state vector representing the market state;

[0007] The state vector is input into a hybrid intelligent optimization model that integrates a learned policy gradient and an improved bald eagle search algorithm to obtain the optimal clearing policy. The improved bald eagle search algorithm generates the initial policy, and the learned policy gradient is used for policy iteration and reinforcement.

[0008] Based on the optimal clearing strategy, the clearing process is automatically executed by a smart contract deployed on the blockchain, and the clearing process and result data are synchronously stored in the blockchain as historical clearing data. The optimal clearing strategy includes a transaction matching priority coefficient, a price adjustment coefficient, and a deviation processing weight.

[0009] The historical clearing data is retrieved from the blockchain, and the core parameters of the hybrid intelligent optimization model are optimized based on the historical clearing data.

[0010] In a preferred embodiment, the improved bald eagle search algorithm generates an initialization strategy, including:

[0011] The core decision parameters that constitute the optimal clearing strategy are encoded into population individuals containing multiple strategy combinations;

[0012] A reverse learning mechanism is used to generate reverse individuals, which are then merged with a random initial population and high-fitness individuals are selected to construct an initial population with high diversity.

[0013] A three-stage optimization mechanism, including search space selection, space exploration, and dive capture, is used to iteratively evolve the population.

[0014] The optimal individuals in the population obtained after iterative evolution are mapped to the initialization parameters of the learning policy gradient.

[0015] As a preferred embodiment, the iterative evolution of the population using a three-stage optimization mechanism comprising search space selection, space exploration, and dive capture includes:

[0016] Based on the best individual with the highest fitness in the current population and the average value of all individuals in the population, other individuals are guided to move in the direction between the best region and the population center region in order to lock in the candidate strategy region that meets the constraints of the distributed power trading scenario through global exploration.

[0017] Within the candidate strategy region, the population individuals are locally optimized by refining the search trajectory, so that the population approaches the optimal solution within the region;

[0018] Based on the current best individual in the population, a step size optimization strategy is introduced to adjust the individual update range, thereby obtaining high-quality individuals in the population that are suitable for distributed power trading needs.

[0019] In a preferred embodiment, the learned policy gradient performs policy iteration and reinforcement, including:

[0020] Based on multi-dimensional data from all scenarios of distributed power trading, a multi-scenario meta-training environment is constructed, which includes state, action, reward and next state transition relationship.

[0021] An objective function is constructed with the goal of maximizing the expected overall transaction return, and regularization terms for policy entropy and prediction vector entropy are introduced into the objective function.

[0022] The meta-gradient is calculated by truncating backpropagation through a sliding window, and the model parameters are iteratively updated using an adaptive optimizer until the change in the objective function value is lower than the preset convergence threshold.

[0023] As a preferred implementation, based on the optimal clearing strategy, the clearing process is automatically executed through smart contracts deployed on the blockchain, including:

[0024] Based on the transaction matching priority coefficient in the optimal clearing strategy, the orders submitted by the power purchaser and the power seller are sorted. The matching constraint is that the power purchase price is not lower than the power sales price, the total cost of network loss and policy-related costs are equal, and the power purchase and sales transaction pairs with better overall benefits are locked in first.

[0025] Based on the price adjustment coefficient in the optimal clearing strategy, and combined with the electricity purchase and sale quotation, the transaction base price is calculated. Then, the transmission and distribution service-related costs, network loss sharing costs, and government funds are added to the transaction base price to determine the final transaction price that meets policy compliance and market fairness.

[0026] The core transaction terms, including the identity information of both parties, the volume of electricity traded, and the final transaction price, will be formalized into an electronic contract.

[0027] The hash value of the electronic contract is calculated and simultaneously stored in the blockchain distributed ledger to form a transaction certificate.

[0028] As a preferred implementation, the clearing process is automatically executed through a smart contract deployed on the blockchain based on the optimal clearing strategy, and also includes a performance deviation handling step:

[0029] By collecting real-time data on the actual output of distributed power sources and the actual electricity consumption of power users through blockchain, and combining this data with the transaction volume agreed in the electronic contract, the deviation volume and deviation ratio of the power seller and the power buyer are calculated respectively.

[0030] When any deviation ratio exceeds the preset allowable deviation threshold, the deviation assessment fee is automatically calculated and the fee settlement is executed through a smart contract based on the deviation processing weight in the optimal clearing strategy, and the settlement result is simultaneously stored in the blockchain distributed ledger.

[0031] When the deviation ratio is within the preset allowable deviation threshold, the deviation data and related performance records are synchronously stored on the blockchain.

[0032] In a preferred embodiment, the step of optimizing the core parameters of the hybrid intelligent optimization model based on the historical clearing data includes:

[0033] Retrieve the historical clearing data from the blockchain, and construct a feedback feature vector representing the transaction execution effect and market changes based on the historical clearing data;

[0034] Based on the feedback feature vector, the objective function of the learning policy gradient is recalibrated, and the model learning rate and iterative convergence conditions are adjusted.

[0035] Based on market fluctuations, the control parameters related to the global exploration range and local convergence speed of the improved bald eagle search algorithm are dynamically adjusted.

[0036] As a preferred implementation, the clearing process and result data are synchronously stored in the blockchain as historical clearing data, including:

[0037] The clearing process and result data are organized using a tree-structured hash data structure, and the hash values ​​of the data are stored in the blockchain block.

[0038] Sensitive data, including transaction quotes and actual electricity consumption, are encrypted using a symmetric encryption algorithm, and the key of the symmetric encryption algorithm is managed in conjunction with asymmetric encryption technology to achieve data privacy protection.

[0039] In a preferred embodiment, the automatic execution of the clearing process via smart contracts deployed on the blockchain further includes a consensus verification step for the clearing result, specifically including:

[0040] A consensus cluster composed of transaction center nodes and distributed heavy nodes uses the distributed consensus algorithm preset by the blockchain network to verify the consistency of the clearing results output by the smart contract.

[0041] When the number of verified consensus nodes reaches a preset ratio, the clearing result is finally confirmed and recorded in the blockchain.

[0042] Secondly, embodiments of the present invention also provide a distributed power trading intelligent clearing and optimization system based on deep reinforcement learning, comprising:

[0043] The state feature extraction module is used to collect multi-dimensional data of the entire scenario of distributed power trading, and to preprocess the multi-dimensional data to generate a state vector representing the market state.

[0044] The hybrid intelligent optimization module is used to input the state vector into a hybrid intelligent optimization model that integrates a learned policy gradient and an improved bald eagle search algorithm to obtain the optimal clearing policy. The improved bald eagle search algorithm generates the initialization policy, and the learned policy gradient is used for policy iteration and reinforcement.

[0045] The intelligent clearing execution module is used to automatically execute the clearing process through smart contracts deployed on the blockchain, and synchronously store the clearing process and result data as historical clearing data in the blockchain. The optimal clearing strategy includes transaction matching priority rules, price adjustment rules, and deviation assessment rules.

[0046] The feedback optimization module is used to retrieve the historical clearing data from the blockchain and perform feedback optimization on the core parameters of the hybrid intelligent optimization model based on the historical clearing data.

[0047] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0048] One or more processors;

[0049] Storage device for storing one or more programs;

[0050] When the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent clearing and optimization method for distributed power trading based on deep reinforcement learning as described in any embodiment of the present invention.

[0051] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the intelligent clearing and optimization method for distributed power trading based on deep reinforcement learning described in any embodiment of the present invention.

[0052] The present invention achieves the following beneficial effects:

[0053] (1) This invention integrates deep reinforcement learning, metaheuristic optimization and blockchain technology. In view of the characteristics of distributed power trading, such as multiple participants, high frequency, complex data and strong constraints, it constructs a closed-loop technology system of data on-chain - intelligent learning - automated clearing - dynamic feedback, which realizes the efficiency, fairness and credibility of transaction clearing.

[0054] (2) This invention proposes a learning-based policy gradient (LPG) meta-learning framework, which breaks through the limitations of traditional reinforcement learning that relies on manually designed value functions and temporal difference learning and other core components. It realizes the autonomous discovery of complete clearing policy update rules from transaction data and adapts to dynamic market environment and complex constraints. This invention proposes an improved bald eagle search optimization algorithm (IBES), which integrates reverse learning, Lévy flight and nonlinear control parameters to improve the global coverage and local convergence speed of policy search and solve the problem that basic metaheuristic algorithms are prone to getting trapped in local optima.

[0055] (3) This invention integrates the learning policy gradient (LPG) and the improved bald eagle search (IBES) algorithm to construct a hybrid mode of metaheuristic optimization initialization + reinforcement learning iterative optimization, so as to realize the autonomous learning and global optimization of the clearing strategy and balance the multi-objective optimization needs.

[0056] (4) This invention combines blockchain technology to achieve trusted storage of transaction data, transparent and traceable clearing rules, and multi-node consensus verification, thereby solving the lack of trust and security risks in distributed transactions and ensuring the fairness and reliability of transactions. Attached Figure Description

[0057] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0058] Figure 1 This is a flowchart of the intelligent clearing and optimization method for distributed power trading based on deep reinforcement learning provided in an embodiment of the present invention;

[0059] Figure 2 This is a framework diagram of the intelligent clearing and optimization method for distributed power trading based on deep reinforcement learning provided in the embodiments of the present invention;

[0060] Figure 3 This is an application deployment diagram of the distributed power trading intelligent clearing and optimization method based on deep reinforcement learning provided in the embodiments of the present invention;

[0061] Figure 4 This is a schematic diagram of the intelligent clearing and optimization method for distributed power trading based on deep reinforcement learning provided in an embodiment of the present invention;

[0062] Figure 5 This is a schematic diagram of the structure of the distributed power trading intelligent clearing and optimization system based on deep reinforcement learning provided in an embodiment of the present invention;

[0063] Figure 6This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0064] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0065] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations (or steps) can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the figures. The process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0066] Example 1

[0067] like Figure 1 The diagram shows a flowchart of a distributed power trading intelligent clearing and optimization method 100 based on deep reinforcement learning provided in Embodiment 1 of the present invention. The method 100 specifically includes the following steps:

[0068] S110. Collect multi-dimensional data of the entire scenario of distributed power trading, and preprocess the multi-dimensional data to generate a state vector representing the market state.

[0069] Specifically, in combination Figure 4 It can be seen that the above-mentioned multi-dimensional data of the entire scenario of distributed power trading includes market participant characteristics, trading market characteristics, network constraint characteristics, and policy and environmental characteristics, etc.

[0070] Among these, the characteristics of the aforementioned market entities include: the installed capacity of distributed generation enterprises. (Unit: MW) Power output forecast (Unit: MWh), Historical Price Records (Unit: Yuan / MWh), Energy-saving and environmental protection indicators (e.g., carbon emission coefficient per unit of electricity), credit rating (0-100 points); Electricity load forecasting for power users (Unit: MWh), Historical Electricity Purchase Preferences (e.g., peak-valley electricity purchase ratio), account margin status (Unit: Yuan) and power reliability requirements (e.g., the allowed power outage duration).

[0071] The characteristics of the trading market include: the total amount of electricity purchase applications submitted during the current period. Total electricity sales declaration Distribution of electricity purchase prices Electricity sales price distribution Remaining time of transaction declaration window (Unit: min) Number of unmatched orders Historical transaction price trends (e.g., the average price change rate over the past three periods).

[0072] Network constraints include: upper limit of distribution network transmission capacity. (Unit: MW), Allowable deviation range of node voltage (like Network loss coefficient (like ), Power transmission channel occupancy status idle, (Occupied).

[0073] Policy and environmental characteristics include: transmission and distribution pricing standards (Unit: Yuan / MWh), Government Funds and Surcharges (Unit: Yuan / ) Environmental protection subsidy coefficient (e.g., 0.03 yuan / MWh), deviation assessment threshold (like ).

[0074] In some embodiments, to eliminate the influence of dimensions, the aforementioned multidimensional data is further preprocessed by using min-max normalization to map the range of the multidimensional data to... Within the interval, the final output dimension is (like ) state vector As input to the hybrid intelligent optimization module, where, : The dimension of the state vector The state space is the vector of all possible states. A set of.

[0075] S120. Input the state vector into a hybrid intelligent optimization model that integrates a learned policy gradient and an improved bald eagle search algorithm to obtain the optimal clearing policy. The improved bald eagle search algorithm generates the initialization policy, and the learned policy gradient is used for policy iteration and reinforcement.

[0076] In some embodiments, the present invention integrates the Learned Policy Gradient (LPG) and the Improved Bald Eagle Search (IBES) algorithm, and achieves autonomous learning and global optimization of the clearing policy through a hybrid mode of metaheuristic optimization initialization and reinforcement learning iterative optimization. Specifically, it includes the following steps:

[0077] (1) Reinforcement learning optimization based on learning policy gradient (LPG)

[0078] Learning-based policy gradients (LPG) overcome the limitations of traditional reinforcement learning, which relies on manually designed value functions and temporal difference learning. LPG can autonomously discover complete clearing strategy update rules from trading data, specifically including:

[0079] (1.1) Hybrid Intelligent Optimization Model Architecture Design

[0080] Input layer: Receives the state vector output from step S110. It is mapped to the hidden layer through the fully connected layer;

[0081] Hidden layers: A two-layer fully connected network with 64 neurons in each layer is used, and the ReLU activation function is employed to enhance the model's nonlinear fitting ability.

[0082] Output layer: The model output contains policy vectors. (Mapping clearing actions, such as trade matching priority and price adjustment coefficient) and prediction vector (Self-learning semantics related to trading value, such as expected future returns and risk coefficients); the output layer will include the strategy vector. Mapped to via the Sigmoid function , predict vector The probability distribution constraint is satisfied by the Softmax function. .

[0083] (1.2) Constructing a meta-training mechanism

[0084] (1.2.1) Training environment construction

[0085] Based on the aforementioned multi-dimensional data from the entire distributed power trading scenario, a meta-training environment is constructed. Each environment sample contains states. Clearing actions (e.g., matching decisions, price calculations), rewards (Such as transaction efficiency rewards, cost-saving rewards), next status .

[0086] (1.2.2) Constructing the objective function

[0087] An objective function is constructed with the goal of maximizing the expected overall return of the transaction. Regularization terms for policy entropy and prediction vector entropy are introduced into the objective function. The formula is:

[0088]

[0089] in, Meta-parameters of the model refer to all trainable parameters of the LPG model, including the weights and biases of fully connected layers, and are the core objects of model learning and optimization. The mathematical expectation operator is used to calculate the mean of a random variable; here, it represents the mean of the state space. and action distribution The expected value is to eliminate the influence of random factors on the objective function; State space The probability distribution; Policy distribution refers to the distribution of the model's meta-parameters. ,state Output clearing action The probability distribution reflects the model's preference for different actions; : Trading cycle steps, refers to the number of time steps within a complete trading cycle; : Time step index, with a value range of 0 to ; Discount factor, with a value range of: , The closer the value is to 1, the more the model values ​​the long-term benefits of future steps (such as the performance stability of subsequent transactions). The closer it is to 0, the more the model focuses on the immediate benefits of the current step (such as the current transaction efficiency). : No. The reward signal for the timing step.

[0090] (1.2.3) Meta-gradient Calculation

[0091] The elementary gradient is calculated by truncating backpropagation using a sliding window. The formula is:

[0092] in, The proportional sign indicates that the left-hand gradient and the right-hand expression are in the same direction. : Policy log-likelihood with respect to metaparameters The gradient is used to measure the impact of changes in meta-parameters on the probability of action selection. Entropy function, used to measure the uncertainty of a probability distribution: Policy vector The entropy of a model is such that a larger entropy value indicates a higher degree of uncertainty in the model's action selection and a stronger exploratory nature. Prediction vector The higher the entropy value, the more fully the model explores the semantics of transaction value, avoiding being limited to a single semantic understanding; Regularization coefficient, with values ​​ranging from 0 to 10. Used to balance the core terms and regularization terms of the objective function: : Policy entropy regularization coefficient, which adjusts the weight of policy exploration diversity; Predict the vector entropy regularization coefficient to adjust the weight of semantic exploration of transaction value; Policy vector Regularization coefficients are used to suppress policy vectors. The magnitude of the amplitude should be adjusted to avoid overly aggressive model updates. Prediction vector Regularization coefficients are used to suppress the prediction vector. The amplitude should be adjusted to avoid overfitting. Policy vector of Norm square, calculated as follows (because (A one-dimensional vector) is used for regularization constraints to prevent the model from becoming unstable due to excessively large parameters. Prediction vector of Norm square, calculated as follows This is used for regularization constraints to prevent overfitting caused by excessively large predicted vector magnitudes. Strategy Entropy Regarding meta-parameters The gradient is used to guide the model to maintain a certain level of exploration diversity during the update process; Predicted vector entropy Regarding meta-parameters The gradient is used to guide the model to continue exploring the semantics of transaction value during the update process; Policy vector Norm square with respect to the elementary parameter The gradient is used to implement regularized updates of the policy vector, suppressing magnitude growth; Prediction vector Norm square with respect to the elementary parameter The gradient is used to implement regularized updates of the prediction vector, suppressing magnitude growth.

[0093] (1.2.4) Parameter Update

[0094] An adaptive optimizer is used to iteratively update the model parameters until the change in the objective function value is lower than a preset convergence threshold. Specifically, in this embodiment, the Adam optimizer is used to update the meta-parameters. Learning rate Iterate until the objective function converges (e.g., after 100 consecutive iterations). .

[0095] in, The difference in the objective function between two adjacent iterations is calculated as follows: ( (This represents the current iteration round), reflecting the magnitude of change in the objective function.

[0096] (1.3) Output of the optimal clearing strategy

[0097] After the model is trained, for the input state Output the optimal clearing strategy, which includes a transaction matching priority coefficient. Price adjustment coefficient and deviation handling weight .

[0098] in, The range of values ​​is It is used to adjust the priority weights (such as the impact of environmental indicators and credit ratings) when matching transactions. The closer the value is to 1, the greater the impact of non-price factors on matching priority.

[0099] Price adjustment coefficient, with a range of values ​​of [value range missing]. It is used to balance the proportion of market quotations and public costs in the transaction price, and is the core weighting parameter for calculating the transaction price.

[0100] : Deviation handling weight, with a value range of This is used to determine the assessment intensity of the deviation in electrical quantity. The larger the value, the greater the penalty for both parties in a transaction that exceeds the deviation threshold.

[0101] (2) Metaheuristic optimization based on improved bald eagle search (IBES)

[0102] This invention integrates back learning, Lévy flight, and nonlinear control parameters into the basic Bald Eagle Search (BES) algorithm to improve the global coverage and local convergence speed of policy search, providing high-quality initialization parameters for the LPG model. The invention includes the following steps:

[0103] (2.1) Population initialization

[0104] (2.1.1) Population Design

[0105] Encode the core decision parameters that constitute the optimal clearing strategy (such as...) ) Encoding as population individuals Population size .

[0106] in, : No. The transaction matching priority coefficient for each individual is Candidate values; : No. The price adjustment coefficient for each individual is Candidate values; : No. The bias treatment weight for each individual is Candidate values.

[0107] (2.1.2) Reverse learning initialization

[0108] For the randomly generated initial population Generate reverse individuals Individuals with high fitness are selected to form the initial population, thereby improving population diversity.

[0109] in, : The upper limit of parameter values, that is, the maximum allowed value of the clearing strategy parameters, specifically: ; The lower limit of parameter values, i.e., the minimum allowed value of the clearing strategy parameter, is as follows: .

[0110] Selecting individuals with high fitness: After merging the initial population with the reversed individuals, calculate the fitness of each individual and retain those with the highest fitness ranking. Individuals form a new initial population, which both expands the search scope and ensures the quality of the initial population.

[0111] (2.2) Three-stage optimization mechanism

[0112] (2.2.1) Search Space Selection Phase (Global Exploration)

[0113] The core objective of this stage is to screen the search region where potential optimal clearing strategies reside, expand the global exploration scope, avoid missing optimal solutions, and lay the foundation for subsequent local searches. The formula is:

[0114] ;

[0115] ;

[0116] ;

[0117] in, The current iteration number starts from 0 and increments until the maximum iteration number is reached. ; Maximum number of iterations refers to the maximum number of rounds of population evolution (e.g., ...). ), balancing optimization effectiveness and computational cost; : No. The updated position of each individual, i.e., the combination of clearing strategy parameters adjusted after global exploration. ; The individual with the highest fitness in the current population corresponds to the current optimal purging strategy, with the following parameter combination: It is the core guiding principle for global exploration; The average individual in the population integrates the historical search information of all individuals in the current population to guide individuals to move closer to the optimal area of ​​the population. Nonlinear control parameters, value range It decays linearly with the number of iterations, especially in the early stages of iteration. Approaching 2 enhances the global exploration scope; later in the iteration. Approaching 1.5, gradually narrow the search range to balance exploration and convergence; Uniform random number, range of values .

[0118] The overall trading fitness is used to assess the merits of individuals. The calculation formula is:

[0119] Transaction rate Price deviation rate Network loss rate

[0120] Among them, the transaction rate is the first weighted term of the fitness function, and it is calculated as: transaction rate

[0121] Price deviation rate: The second weighted term of the fitness function, calculated as: Price deviation rate , ( A higher price deviation rate indicates a fairer and more stable pricing strategy.

[0122] Network loss rate: The third weighted term in the fitness function, calculated as: Network loss rate The larger the (1-network loss rate) is, the better the strategy controls the network loss.

[0123] The weight coefficients of the fitness function, where the weight coefficients satisfy... It is used to adjust the importance ratio of transaction rate, price stability, and network loss control, highlighting the core position of transaction efficiency.

[0124] (2.2.2) Space exploration phase (local fine search)

[0125] The core objective of this stage is to perform a fine scan using a spiral trajectory within the area determined in the search space selection stage, accurately locate the local optimal clearing strategy, and improve the optimization accuracy.

[0126] (a) The polar coordinate parameters are calculated as follows:

[0127] ;

[0128] ;

[0129] ;

[0130] Where rand is a uniform random number generator function, and the generation interval is... Random numbers within; : No. The individual spiral rotation angle is used to control the rotation direction and angle of the spiral trajectory; : Helical rotation angle control parameter, value range As the number of iterations increases linearly, in the early stages of iteration... Approaching 5, the spiral rotation is gentle, allowing for a wide range of exploration; in the later stages of iteration... Approaching 10, the spiral rotations become denser, allowing for more precise local searches; : No. The spiral radius of each individual is used to control the coverage area of ​​the spiral trajectory; : Number of spiral turns control parameter, value range It decreases linearly with the number of iterations, especially in the early stages of iteration. Approaching 2, with a high number of spiral turns, and a wide exploration range; later stages of iteration Approaching 0.5, with fewer spiral turns, it focuses on local optima.

[0131] (b) Normalized polar coordinate components:

[0132]

[0133] in,

[0134] : No. Polar coordinates of an individual Components, calculated as follows It is a sine function, and Together they form the polar coordinate representation of the spiral trajectory;

[0135] : No. Polar coordinates of an individual Components, calculated as follows It is a cosine function;

[0136] Polar coordinates The maximum absolute value of the component, i.e., all individuals The maximum value among absolute values ​​is used to compare... Perform normalization processing;

[0137] Polar coordinates The maximum absolute value of the component, i.e., all individuals The maximum value among absolute values ​​is used to compare... Perform normalization processing;

[0138] Normalized polar coordinates Components, range of values Used to control individuals in Local search step size in the direction;

[0139] Normalized polar coordinates Components, range of values Used to control individuals in The local search step size in the direction.

[0140] (c) Location update:

[0141]

[0142] in, : No. The adjacent individuals of an individual, i.e., those with the index in the population. individuals (if) ,but This is used to enhance the relevance of local searches and avoid isolated searches of individuals; : No. The updated position of each individual during the local detection phase, i.e., the combination of clearing strategy parameters after fine-tuning the spiral trajectory search.

[0143] (2.2.3) Dive-and-capture phase (local convergence)

[0144] The core objective of this phase is to move towards the current optimal strategy. A fast approximation method achieves local convergence, while Lévy flight is introduced to avoid getting trapped in local optima, outputting the final high-quality initialization parameters, as shown in the formula:

[0145] (a) Levy's flight stride:

[0146]

[0147]

[0148]

[0149]

[0150] in, Levy's flight stride is used to increase long-distance jump capability and avoid local optima. A random variable that follows a standard normal distribution, i.e. This represents a normal distribution with a mean of 0 and a variance of 1, used to generate the random step size for the Levy flight. Lévy distribution shape parameter, range of values The distribution characteristics of Levi's flight stride control determine the probability of long-distance jumps; : Scale parameters of Levy's flight stride; For the gamma function (an extension of the factorial function, such as...) ), used to standardize Levy's flight stride.

[0151] (b) Location update

[0152]

[0153]

[0154] in, Normalized bipolar coordinates Components, range of values ; Normalized bipolar coordinates Components, range of values Same function ; : Approaching weighting coefficients, range of values Used to control the population mean of individuals. and the optimal individual The intensity of their approach. The larger the value, the more it tends to be closer to the population mean. The larger the value, the more likely it is to be the optimal individual. : Randomly weighted optimal individual term, adjusted by random numbers The degree of impact on individual updates, balancing convergence speed and exploration diversity; : No. The final updated position of each individual during the dive capture phase, i.e. the optimal clearing strategy parameter combination obtained by the IBES algorithm optimization, is used as the initialization parameter of the LPG model.

[0155] (2.2.3) Integration with Learning Policy Gradient (LPG)

[0156] The optimal individual after IBES optimization As initialization parameters for the LPG model This allows the initial LPG policy to have high fitness, shortening the convergence time of reinforcement learning (e.g., reducing the number of convergence iterations). ).

[0157] The optimal individual after IBES algorithm optimization, i.e., the globally optimal clearing strategy parameter combination obtained through a three-stage optimization mechanism, is specifically... The optimal candidate values ​​for the corresponding transaction matching priority coefficient, price adjustment coefficient, and deviation handling weight; Initialization meta-parameters of the LPG model refer to the initial weights, biases, and other parameters of the fully connected layers of the LPG model, obtained through PIBES-best mapping, replacing the traditional random initialization method; Initial policy fitness: The LPG model uses... The initial fitness (i.e., as an initialization parameter) ),because Having been optimized through IBES to have high adaptability, the initial LPG strategy does not need to be learned from scratch, resulting in better initial performance.

[0158] S130. Based on the optimal clearing strategy, the clearing process is automatically executed through a smart contract deployed on the blockchain, and the clearing process and result data are synchronously stored in the blockchain as historical clearing data. The optimal clearing strategy includes transaction matching priority rules, price adjustment rules, and deviation assessment rules.

[0159] In some embodiments, based on the optimal strategy output by the hybrid intelligent optimization module in step S120 above ( By combining blockchain smart contract technology, the entire process of transaction matching, price calculation, contract generation, and deviation handling can be automated.

[0160] (3.1) Sorting rules

[0161] Based on the priority coefficient of the strategy output The electricity purchaser's quotation shall be based on: quotation Sort the electricity sellers' quotations in descending order according to: Quotation × (1 - Sort in ascending order, where, To meet environmental protection targets, prioritize environmentally friendly power sources. Credit rating is used, with those having higher credit scores receiving higher priority.

[0162] Matching logic: Prioritize matching based on electricity purchase price. Electricity sales price And the trading pair with the highest overall returns, among which,

[0163] Network loss coefficient This refers to the proportion of energy loss during the power transmission process. Transmission and distribution price standard (yuan / MWh): The charging standard for power grid companies to provide transmission and distribution services; Government funds and surcharges (RMB / MWh), special fees collected in accordance with national policies; The cost allocation amount corresponding to network loss, i.e., the additional cost incurred due to transmission loss; electricity sales price. The lowest acceptable transaction price for the electricity seller; the highest overall return: the core objective of matching, the overall return covers the economic and environmental benefits of both parties in the transaction, and priority is given to matching the electricity purchase and sale combination that meets the constraints and has the highest overall return.

[0164] Priority Supplement: When prices are the same, the one submitted the earlier will be given priority ( → Those with excellent energy-saving and environmental protection indicators high Electricity allocation ratio ( The order of transactions determines the priority of transactions, ensuring fair matching.

[0165] in, : Transaction declaration time refers to the specific timestamp of when market participants submit their electricity purchase / sale declarations. When prices are the same, (Energy conservation and environmental protection indicators): The second priority criterion when both the quoted price and the application time are the same. The better the quality (e.g., the lower the carbon emission coefficient per unit of electricity), the higher the priority for transaction, highlighting the environmental protection orientation; Electricity volume declared by a single market entity (unit: MWh); Total declared electricity volume of the same type (electricity purchase / sale) in the current period (unit: MWh); : The percentage of electricity declared.

[0166] (3.2) Price determination

[0167] (3.2.1) Calculation of basic price

[0168] Price adjustment coefficient based on strategy output Based on the converted electricity purchase and sale price, the formula is:

[0169]

[0170] in, The base price of the transaction is the initial price calculated based on the quotations from both the buyer and seller of electricity. The electricity purchase price declared by the electricity purchaser; The electricity sales price declared by the electricity seller.

[0171] (3.2.2) Cost accumulation

[0172] The final transaction price is determined by adding transmission and distribution fees, grid loss costs, and government funds to the aforementioned base price. The formula is:

[0173] ;

[0174] ;

[0175] in, Public cost weighting coefficient, and price adjustment coefficient Complementary It is used to adjust the proportion of public costs in the final transaction price. The smaller the value, the more significant the impact of public costs. Transmission and distribution price standards; Government funds and surcharges; Network loss costs, namely the additional expenses incurred due to power loss, are shared by both parties to the transaction. The amount of public cost allocation will include network loss costs and government funds in the final transaction price according to their weights, ensuring that public costs are reasonably recovered.

[0176] (3.2.3) Establish price constraints

[0177] Final transaction price Must meet:

[0178] ;

[0179] Among them, the on-grid tariff for distributed power generation refers to the distributed power generation projects approved by the state or local governments; the grid sales tariff refers to the official price set by the grid company when selling electricity to end users.

[0180] Based on the above price constraint logic, we can ensure that the transaction price not only meets policy requirements but also has market rationality.

[0181] (3.3) Contract generation

[0182] Once a transaction is successfully matched (meaning the buyer and seller of electricity meet the above matching logic and achieve the highest overall benefit), the system automatically triggers the smart contract deployed on the blockchain network without manual intervention. This triggers the contract, which includes the identities (blockchain addresses) of both parties and the amount of electricity transacted. Deviation handling rules (exceeding) Press at time Charge fees, The core transaction terms (for bias handling weights) are solidified into electronic contracts.

[0183] Calculate the contract hash value based on the electronic contract. (Contract content), and will Key contract information is simultaneously stored on a blockchain distributed ledger to form transaction vouchers, ensuring immutability and traceability.

[0184] (3.4) Deviation handling

[0185] Real-time monitoring of distributed power output using blockchain technology Based on the transaction volume agreed upon in the aforementioned electronic contract Calculate the deviation in electricity volume for each electricity seller. Electricity seller deviation ratio The deviation in electricity consumption and the deviation ratio of the electricity purchaser. :

[0186] ;

[0187] ;

[0188] When the electricity seller's deviation ratio At that time, the smart contract automatically implements the deviation handling strategy for the electricity seller, based on the deviation fee. The costs are calculated, with the electricity seller paying the costs and the income and expenditure records stored in the blockchain in real time.

[0189] When the deviation ratio of the electricity purchaser At that time, the smart contract automatically implements the deviation handling strategy for the electricity purchaser, based on the deviation fee. The costs are calculated, with the costs paid by the electricity purchaser, and the income and expenditure records are stored in the blockchain distributed ledger in real time.

[0190] When the deviation ratio of the electricity seller / the deviation ratio of the electricity buyer At that time, no deviation fee is charged; only deviation data is recorded for subsequent model feedback.

[0191] S140. Retrieve the historical clearing data from the blockchain and optimize the core parameters of the hybrid intelligent optimization model based on the historical clearing data.

[0192] In some embodiments, the core parameters of the hybrid intelligent optimization model are adjusted based on historical clearing data. The system will be dynamically adjusted and optimized to adapt to market fluctuations and policy changes.

[0193] in, Meta-parameters of the LPG model; The nonlinear control parameters of the IBES algorithm ([1.5,2]) are used to adjust the global exploration range; The spiral rotation angle control parameters ([5,10]) of the IBES algorithm are used to adjust the local search precision. The spiral rotation control parameter of the IBES algorithm , used to adjust the local search range.

[0194] (4.1) Data feedback and feature extraction

[0195] (4.1.1) Feedback Data Collection

[0196] The blockchain network collects clearing results data, market status data, policy parameter changes, and anomaly data in real time.

[0197] Clearing results data: Transaction rate .

[0198] Market State Data: New State Vector (including the updated version) );in, Updated total electricity purchase declaration; Updated total electricity sales declaration;

[0199] Updated distribution of electricity purchase prices; Updated electricity price distribution;

[0200] Policy parameter changes: such as transmission and distribution prices Adjustments, government funds and surcharges Updates and other policy changes directly affect the cost accounting logic of clearing strategies.

[0201] Abnormal data: Node information and behavior records of malicious pricing (such as pricing far exceeding a reasonable range) and data tampering (such as failure of HAR verification).

[0202] (4.1.2) Construction of Feedback Features

[0203] After standardizing the feedback data, construct the feedback feature vector. , as input for model updates; where, The updated state vector of Each component.

[0204] (4.2) Update of Hybrid Intelligent Optimization Module

[0205] Based on the above feedback characteristics Recalculate the objective function of the LPG model. Adjusting the reward function weights, such as the learning rate of the LPG model when there is a market supply and demand imbalance. ,in, The initial learning rate is set to 0.01 to ensure that the model has strong adjustment capabilities in the early stages of updates.

[0206] Adjust the parameters of the IBES model based on feedback data, when When prices fluctuate greatly, increase Enhance global exploration; when When matching efficiency is low, reduce (For example, reducing from 2 to 0.5) accelerates convergence.

[0207] Update the model once after each trading cycle (e.g., daily / weekly) to ensure it adapts to market changes.

[0208] (4.3) Abnormal handling and safe clearance

[0209] Malicious nodes and tampering behavior are detected through blockchain identity authentication (such as RSA signature verification) and data verification (such as Merkle tree comparison). When an anomaly is detected, the model automatically switches to a security clearing mode, employing preset basic strategies (such as... , The system performs clearing to avoid transaction interruption, and at the same time records abnormal nodes in the blockchain blacklist, restricting their subsequent trading permissions (such as prohibiting them from quoting prices for 1 month). The blacklist record cannot be tampered with, thus ensuring market order.

[0210] Example 2

[0211] Combination Figure 2 As can be seen, after performing steps such as state feature extraction, hybrid intelligent optimization, intelligent clearing execution and feedback optimization, it also includes blockchain integration design. This second embodiment will provide a detailed explanation of the blockchain integration design, specifically including aspects such as trusted data storage, smart contract contracts, consensus and verification mechanisms.

[0212] Preferably, trusted data storage includes synchronously storing the clearing process and result data as historical clearing data in the blockchain:

[0213] Key data generated throughout the distributed power trading process are synchronously stored in the blockchain as historical clearing data, and the hash value of the data is stored in the blockchain block.

[0214] Sensitive data, including transaction quotes and actual electricity consumption, are encrypted using a symmetric encryption algorithm, and the key of the symmetric encryption algorithm is managed in conjunction with asymmetric encryption technology to achieve data privacy protection.

[0215] The key data generated during the entire distributed power generation transaction process includes, but is not limited to: the state feature vector in step S110. (Hash value), Hybrid optimization parameters of step S120 ( The clearing result of step S130 ( Feedback data from step S140 ( ).

[0216] Storage Structure: Data is organized using a Merkle tree, and the above data is organized using a Merkle tree hash data structure to construct an immutable distributed storage system. Each block contains a standardized structure of a block header and a block body, with the block header recording the hash of the previous block. Merkle root timestamp The hash value of the on-chain data stored in the block body is calculated using the SHA256 algorithm to ensure that the data cannot be tampered with.

[0217] Preferably, smart contract collaboration includes aspects such as contract function embedding, multi-channel parallel processing, and dynamic upgrade mechanisms, among which,

[0218] Contract Function Embedding: The smart contract is embedded into the three-stage optimization formula of the IBES algorithm, the reward function and policy gradient calculation rules of the LPG framework, solidifying the business logic of the clearing process and ensuring that the clearing rules are transparent and traceable.

[0219] Multi-channel parallel processing: Isolated blockchain channels are constructed based on transaction type (bilateral negotiation, centralized bidding, listing trading). Each channel independently stores corresponding data, and the Kafka consensus algorithm is used within the channel (tolerance). Node failure), improve concurrent processing capabilities (e.g., support 5000 concurrent applications per second);

[0220] Dynamic upgrade mechanism: When the optimization parameters in step S120 are adjusted, the smart contract is updated through the proposal-voting-upgrade mechanism, and the upgrade record is recorded on the chain to ensure traceability.

[0221] Preferably, the consensus and verification mechanism includes building a verification network based on the Kafka consensus algorithm, combined with... Figure 3 It adopts the Kafka consensus algorithm, which is agreed upon by the transaction center and key nodes (such as...). Figure 3 The six key nodes deployed in the Jiangsu Province pilot program form a consensus cluster to verify the consistency of the clearing results. When the number of consensus nodes that have passed verification reaches a preset proportion (such as...), the consensus cluster is established. When a node is reached, the clearing result is finally confirmed and recorded in the blockchain.

[0222] Additionally, any node can query the hash value of the clearing result through the blockchain interface and compare it with the local calculation result, using the following formula: ,like If the verification passes, the transaction will be fair.

[0223] Regulatory agencies can also access full-process data (including optimization logs from step two and contract records from step three) through the blockchain interface to achieve comprehensive auditing, which complies with the requirements of electricity market regulation.

[0224] Based on the above embodiments, the present invention achieves the following beneficial effects:

[0225] (1) This invention integrates the learning-type strategy gradient meta-learning framework, the improved bald eagle search algorithm and blockchain smart contract technology to realize the autonomous optimization and reliable execution of the distributed power trading clearing strategy, and realizes an efficient, fair and reliable intelligent clearing and optimization method.

[0226] (2) This invention proposes an intelligent clearing framework with autonomous evolution capabilities. It innovatively adopts the learning-based policy gradient (LPG) meta-learning framework, which does not require manual pre-setting of value functions and temporal difference rules. It can autonomously discover the clearing strategy update logic from transaction data in multiple scenarios and can dynamically adapt to market supply and demand fluctuations, policy adjustments and network constraint changes. Compared with traditional reinforcement learning algorithms, the policy adaptability is improved by more than 40%. At the same time, the improved Bald Eagle Search Optimization Algorithm (IBES) expands the global search range and accelerates local convergence through multi-policy fusion, effectively avoiding local optimum traps and improving the initial policy adaptability by 35%, providing high-quality initialization parameters for the LPG model.

[0227] (2) This invention constructs a hybrid mode of metaheuristic optimization initialization + reinforcement learning iterative optimization. It not only uses the IBES algorithm to quickly lock the optimal strategy search area, but also uses the LPG model to iteratively optimize the strategy details, so as to achieve a global optimal balance of multiple objectives such as transaction efficiency, price fairness, and grid loss. In practical applications, this hybrid mode increases the transaction rate by 25%-30%, reduces the price deviation rate by 15%-20%, and reduces the grid loss rate by 10%-15%, which is significantly better than the clearing effect of a single optimization algorithm.

[0228] (3) The blockchain technology of this invention runs through the entire transaction process. Key data such as status feature data, model optimization parameters, and clearing results are stored on the chain. The SHA256 hash algorithm and Merkle tree structure are used to ensure that the data is immutable and the entire transaction is traceable. The smart contract embeds hybrid optimization rules and clearing logic to automatically execute transaction matching, price calculation, contract generation and deviation processing without human intervention, thus avoiding the risk of human operation. Multiple nodes verify the consistency of the clearing results based on the Kafka consensus algorithm to ensure that the transaction rules are fair and transparent and the results are fair and credible.

[0229] (4) Based on the trusted data of blockchain, this invention has constructed a dynamic parameter adjustment mechanism, which can optimize the core parameters of LPG and IBES in real time according to changes in market status, maintain the optimal operating state of the system, and automatically switch to the safe clearing mode when abnormalities such as malicious bidding or data tampering are detected, ensuring the continuity of transactions. The system supports various transaction types such as bilateral negotiation, centralized bidding, and listing transactions, and can be adapted to different scales of distributed power trading scenarios. After pilot verification in Jiangsu Province, it can support more than 10,000 users to trade at the same time, and the clearing calculation time of a single transaction does not exceed 10 seconds, meeting the needs of engineering applications.

[0230] Example 2

[0231] Figure 5 This is a schematic diagram of the structure of a distributed power trading intelligent clearing and optimization system based on deep reinforcement learning, as provided in Embodiment 2 of the present invention. Figure 5 As shown, the system includes:

[0232] The state feature extraction module 510 is used to collect multi-dimensional data of the entire scenario of distributed power trading, and to preprocess the multi-dimensional data to generate a state vector representing the market state.

[0233] The hybrid intelligent optimization module 520 is used to input the state vector into a hybrid intelligent optimization model that integrates a learned policy gradient and an improved bald eagle search algorithm to obtain an output optimal clearing policy, wherein the improved bald eagle search algorithm generates an initialization policy; and the learned policy gradient is used for policy iteration and reinforcement.

[0234] The intelligent clearing execution module 530 is used to automatically execute the clearing process through smart contracts deployed on the blockchain, and synchronously store the clearing process and result data as historical clearing data in the blockchain. The optimal clearing strategy includes transaction matching priority rules, price adjustment rules and deviation assessment rules.

[0235] The feedback optimization module 540 is used to retrieve the historical clearing data from the blockchain and perform feedback optimization on the core parameters of the hybrid intelligent optimization model based on the historical clearing data.

[0236] The distributed power trading intelligent clearing and optimization system based on deep reinforcement learning provided in this embodiment of the invention can execute the distributed power trading intelligent clearing and optimization method based on deep reinforcement learning provided in any of the embodiments of the invention above. It has the corresponding functions and beneficial effects of executing the distributed power trading intelligent clearing and optimization method based on deep reinforcement learning. For detailed process, please refer to the relevant operations of the distributed power trading intelligent clearing and optimization method based on deep reinforcement learning in the foregoing embodiments.

[0237] Example 3

[0238] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, and may also represent various forms of mobile devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0239] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0240] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0241] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 executes the dynamic prediction method for disaster-prone processes based on a multiphysics coupling model described above.

[0242] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0243] The above embodiments are merely illustrative examples and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for intelligent clearing and optimization of distributed power source trading based on deep reinforcement learning, characterized in that, include: Collect multi-dimensional data from the entire scenario of distributed power trading, and preprocess the multi-dimensional data to generate a state vector representing the market state; The state vector is input into a hybrid intelligent optimization model that integrates a learned policy gradient and an improved bald eagle search algorithm to obtain the optimal clearing policy. The improved bald eagle search algorithm generates the initial policy, and the learned policy gradient is used for policy iteration and reinforcement. Based on the optimal clearing strategy, the clearing process is automatically executed by a smart contract deployed on the blockchain, and the clearing process and result data are synchronously stored in the blockchain as historical clearing data. The optimal clearing strategy includes a transaction matching priority coefficient, a price adjustment coefficient, and a deviation processing weight. The historical clearing data is retrieved from the blockchain, and the core parameters of the hybrid intelligent optimization model are optimized based on the historical clearing data.

2. The method according to claim 1, characterized in that, The improved bald eagle search algorithm generates an initialization strategy, including: The core decision parameters that constitute the optimal clearing strategy are encoded into population individuals containing multiple strategy combinations; A reverse learning mechanism is used to generate reverse individuals, which are then merged with a random initial population and high-fitness individuals are selected to construct an initial population with high diversity. A three-stage optimization mechanism, including search space selection, space exploration, and dive capture, is used to iteratively evolve the population. The optimal individuals in the population obtained after iterative evolution are mapped to the initialization parameters of the learning policy gradient.

3. The method according to claim 2, characterized in that, The three-stage optimization mechanism, comprising search space selection, space exploration, and dive capture, is used to iteratively evolve the population, including: Based on the best individual with the highest fitness in the current population and the average value of all individuals in the population, other individuals are guided to move in the direction between the best region and the population center region in order to lock in the candidate strategy region that meets the constraints of the distributed power trading scenario through global exploration. Within the candidate strategy region, the population individuals are locally optimized by refining the search trajectory, so that the population approaches the optimal solution within the region; Based on the current best individual in the population, a step size optimization strategy is introduced to adjust the individual update range, thereby obtaining high-quality individuals in the population that are suitable for distributed power trading needs.

4. The method according to claim 1, characterized in that, The learning-based policy gradient includes policy iteration and reinforcement, which includes: Based on multi-dimensional data from all scenarios of distributed power trading, a multi-scenario meta-training environment is constructed, which includes state, clearing action, reward and next state. An objective function is constructed with the goal of maximizing the expected overall transaction return, and regularization terms for policy entropy and prediction vector entropy are introduced into the objective function. The meta-gradient is calculated by truncating backpropagation through a sliding window, and the model parameters are iteratively updated using an adaptive optimizer until the change in the objective function value is lower than the preset convergence threshold.

5. The method according to claim 1, characterized in that, Based on the aforementioned optimal clearing strategy, the clearing process is automatically executed via smart contracts deployed on the blockchain, including: Based on the transaction matching priority coefficient in the optimal clearing strategy, the orders submitted by the power purchaser and the power seller are sorted. The matching constraint is that the power purchase price is not lower than the power sales price, the total cost of network loss and policy-related costs are equal, and the power purchase and sales transaction pairs with better overall benefits are locked in first. Based on the price adjustment coefficient in the optimal clearing strategy, and combined with the electricity purchase and sale quotation, the transaction base price is calculated. Then, the transmission and distribution service-related costs, network loss sharing costs, and government funds are added to the transaction base price to determine the final transaction price that meets policy compliance and market fairness. The core transaction terms, including the identity information of both parties, the volume of electricity traded, and the final transaction price, will be formalized into an electronic contract. The hash value of the electronic contract is calculated and simultaneously stored in the blockchain distributed ledger to form a transaction certificate.

6. The method according to claim 5, characterized in that, Based on the aforementioned optimal clearing strategy, the clearing process is automatically executed through smart contracts deployed on the blockchain, and also includes a performance deviation handling step: By collecting real-time data on the actual output of distributed power sources and the actual electricity consumption of power users through blockchain, and combining this data with the transaction volume agreed in the electronic contract, the deviation volume and deviation ratio of the power seller and the power buyer are calculated respectively. When any deviation ratio exceeds the preset allowable deviation threshold, the deviation assessment fee is automatically calculated and the fee settlement is executed through a smart contract based on the deviation processing weight in the optimal clearing strategy, and the settlement result is simultaneously stored in the blockchain distributed ledger. When the deviation ratio is within the preset allowable deviation threshold, the deviation data and related performance records are synchronously stored on the blockchain.

7. The method according to claim 1, characterized in that, The step of optimizing the core parameters of the hybrid intelligent optimization model based on the historical clearing data includes: Retrieve the historical clearing data from the blockchain, and construct a feedback feature vector representing the transaction execution effect and market changes based on the historical clearing data; Based on the feedback feature vector, the objective function of the learning policy gradient is recalibrated, and the model learning rate and iterative convergence conditions are adjusted. Based on market fluctuations, the control parameters related to the global exploration range and local convergence speed of the improved bald eagle search algorithm are dynamically adjusted.

8. The method according to claim 7, characterized in that, The clearing process and results data will be synchronously stored on the blockchain as historical clearing data, including: The clearing process and result data are organized using a tree-structured hash data structure, and the hash values ​​of the data are stored in the blockchain block. Sensitive data, including transaction quotes and actual electricity consumption, are encrypted using a symmetric encryption algorithm, and the key of the symmetric encryption algorithm is managed in conjunction with asymmetric encryption technology to achieve data privacy protection.

9. The method according to claim 5, characterized in that, The automated clearing process via smart contracts deployed on the blockchain also includes a consensus verification step for the clearing results, specifically including: A consensus cluster composed of transaction center nodes and distributed heavy nodes uses the distributed consensus algorithm preset by the blockchain network to verify the consistency of the clearing results output by the smart contract. When the number of verified consensus nodes reaches a preset ratio, the clearing result is finally confirmed and recorded in the blockchain.

10. A system for intelligent clearing and optimization of distributed power source trading based on deep reinforcement learning, characterized in that, include: The state feature extraction module is used to collect multi-dimensional data of the entire scenario of distributed power trading, and to preprocess the multi-dimensional data to generate a state vector representing the market state. The hybrid intelligent optimization module is used to input the state vector into a hybrid intelligent optimization model that integrates a learned policy gradient and an improved bald eagle search algorithm to obtain the optimal clearing policy. The improved bald eagle search algorithm generates the initialization policy, and the learned policy gradient is used for policy iteration and reinforcement. The intelligent clearing execution module is used to automatically execute the clearing process through smart contracts deployed on the blockchain, and synchronously store the clearing process and result data as historical clearing data in the blockchain. The optimal clearing strategy includes transaction matching priority rules, price adjustment rules, and deviation assessment rules. The feedback optimization module is used to retrieve the historical clearing data from the blockchain and perform feedback optimization on the core parameters of the hybrid intelligent optimization model based on the historical clearing data.

Images