Microgrid P2P electric energy transaction bidding and matching optimization method

Abstract

The invention relates to the technical field of smart power grids and energy transaction, and discloses a micro-grid P2P electric energy transaction bidding and matching optimization method, which comprises the following steps of: constructing a micro-grid bilateral P2P electric energy transaction market model, defining a buyer, a consumer, as an energy-deficient micro-grid and a seller, as an energy-rich micro-grid, and establishing a micro-grid bilateral P2P electric energy transaction market model; designing a transaction matching and settlement mechanism based on the second price auction mechanism; modeling a transaction environment in the market model as a Markov decision process, introducing a multi-agent reinforcement learning algorithm under a Markov decision process framework, and learning and generating a dynamic bidding strategy oriented to multiple rounds of transactions; in the market matching stage, the dynamic bidding strategy of the producer and the consumer is input into the microgrid bilateral P2P electric energy trading market model, a settlement list is output, the problems that the bidding strategy is static and the market efficiency is low in a traditional trading method are solved, and the method has high adaptability and generalization ability and is suitable for scenes such as intelligent power grid demand side management and energy internet.

Description

Technical Field

[0001] This invention relates to the field of smart grid and energy trading technology, and in particular to a microgrid P2P electricity trading bidding and matching optimization method. Background Technology

[0002] Against the backdrop of rapid development of renewable energy and the continuous advancement of the energy internet concept, microgrids, as important carriers integrating distributed power sources, energy storage devices, and adjustable loads, are gradually evolving from traditional passive access units into "prosumer" nodes with autonomous dispatch and energy management capabilities. The large-scale integration of distributed photovoltaic, wind power, and user-side energy storage has made localized electricity trading based on the distribution network one of the key technological paths to improve renewable energy absorption rates, alleviate distribution network pressure, and enhance energy efficiency. In this process, peer-to-peer (P2P) electricity trading models, due to their decentralization, high flexibility, and ability to reflect individual preferences, have become an important development direction for energy management at the microgrid and distribution network levels.

[0003] Existing electricity trading mechanisms mostly adopt centralized pricing and settlement models led by grid operators or centralized trading institutions, such as matching methods based on time-of-use pricing, unified retail electricity prices, or centralized optimized dispatch. These methods have certain advantages in achieving overall balance and meeting security constraints, but they generally suffer from the following shortcomings: First, price signals are relatively coarse, making it difficult to accurately reflect the congestion situation of local networks and the differences in marginal costs and utility among different microgrid producers and consumers; second, trading strategies are mostly static or pre-set, unable to adaptively adjust according to real-time supply and demand changes and uncertainties (such as renewable energy fluctuations and load fluctuations); third, centralized optimization becomes computationally complex with increasing participant scale, making it difficult to meet the demands of high-frequency, near-real-time distributed markets.

[0004] To address the aforementioned issues, in recent years, academia and engineering practice have begun to introduce methods such as bilateral auctions, game theory modeling, and heuristic optimization to construct P2P electricity trading mechanisms. These mechanisms achieve decentralized matching of local electricity transactions through simultaneous bidding and quoting by both buyers and sellers. Some works have designed matching and pricing rules with incentive compatibility and certain efficiency guarantees based on second-price auctions, bilateral auctions, or McAfee-like mechanisms, which can theoretically improve social welfare and suppress manipulative bidding behavior. However, in most existing studies, the bidding strategies of prosumers still rely on preset rules or simple empirical functions, lacking the ability to self-learn and adaptively optimize in complex dynamic environments. This results in the trading mechanisms not performing ideally in practical applications facing multiple scenarios and types of prosumers. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a microgrid P2P electricity trading bidding and matching optimization method, which solves the problems of static bidding strategies and low market efficiency in traditional trading methods. It has high adaptability and generalization ability and is applicable to scenarios such as smart grid demand-side management and energy internet.

[0006] The first aspect of this invention provides a microgrid P2P electricity trading bidding and matching optimization method, comprising the following steps: A bilateral P2P electricity trading market model for microgrids is constructed, defining the buyer prosumer as an energy-deficient microgrid and the seller prosumer as an energy-rich microgrid, and designing a transaction matching and settlement mechanism based on a second-price auction mechanism; The trading environment in the market model is modeled as a Markov decision process. A multi-agent reinforcement learning algorithm is introduced under the Markov decision process framework to learn and generate dynamic bidding strategies for multi-round transactions. The dynamic bidding strategies include: electricity price, electricity demand and electricity supply. During the market matching phase, the dynamic bidding strategy of producers and consumers is input into the microgrid bilateral P2P power trading market model, and a settlement list is output.

[0007] Optionally, constructing a microgrid bilateral P2P electricity trading market model includes the following steps: Microgrid units are defined as electricity producers and consumers participating in the market. Among them, microgrids with a positive electricity deficit that need to purchase electricity from external sources are defined as energy-deficient microgrid units, which are buyers and consumers. Microgrid units with a surplus of electricity that can sell electricity to external sources are defined as energy-rich microgrids, which are sellers and consumers. Based on the above definition and using a distributed architecture, each prosumer independently submits bidding information, including electricity quotes, through a proxy node. and battery level The market platform aggregates all bidding information within each discrete trading cycle, forming a bilateral P2P electricity trading market.

[0008] Optionally, the design of a transaction matching and settlement mechanism based on a second-price auction mechanism includes: With the goal of maximizing social welfare, an objective function and its constraints are established to optimize the allocation of electricity transactions between buyers and sellers, thereby maximizing the social welfare of the system while meeting the power supply capacity and electricity demand constraints of each microgrid. In the process of market modeling, a probabilistic model is introduced to predict the behavior of prosumers, and historical data is used to train a generative adversarial network to simulate the bidding distribution. By adopting a hierarchical market structure, the large-scale market is decomposed into multiple sub-communities, and clustering algorithms are used to group prosumers and consumers, enabling local P2P transactions to take place within each sub-community.

[0009] Optionally, the trading environment in the market model is modeled as a Markov decision process. A multi-agent reinforcement learning algorithm is introduced within the Markov decision process framework to learn and generate dynamic bidding strategies for multi-round trading, including the following steps: The P2P electricity trading market based on the second-price auction mechanism is formalized as a Markov decision process. The state space contains global market information and individual characteristics, the action space contains the actions of each producer and consumer, and the environmental state transition in the state transition dynamics is driven by the McAfee rule. The state transition results are generated by clarifying the processing flow of the second-price auction mechanism in each round of trading. The second-price auction mechanism is integrated with a multi-agent reinforcement learning algorithm to achieve joint learning and updating of agent bidding strategies. The multi-agent reinforcement learning algorithm adopts a multi-agent deep deterministic policy gradient framework, in which the input of the critic network includes the state transition results generated by the second-price auction mechanism.

[0010] Optionally, by inputting the dynamic bidding strategy of prosumers into the microgrid bilateral P2P electricity trading market model, the settlement list is output, including the following steps: Bidding information preprocessing and verification: Collect all bidding information submitted by producers and consumers, verify the validity of the bids, and remove outliers; After processing through the second-price auction mechanism, the transaction results are output: the matching result matrix of the transaction volume of each buyer and seller prosumer is output, and a settlement list is generated, which includes the transaction volume, settlement price and transaction revenue of each prosumer.

[0011] Optionally, it also includes simulating and verifying the microgrid bilateral P2P electricity trading market model, dynamic bidding strategy, and transaction matching and settlement mechanism through a simulation platform, including the following steps: The electricity consumption or production of prosumers is simulated and generated through a prosumer agent server, and a dynamic bidding strategy for each prosumer is generated based on a multi-agent reinforcement learning algorithm and transmitted to the local server. On a local server, the dynamic bidding strategy is input into the transaction matching and settlement mechanism based on the second-price auction mechanism to complete one or more transaction processes, and the transaction results of each round of transactions are obtained, thereby verifying the efficiency of the transaction mechanism and the dynamic bidding strategy. The simulation platform continuously simulates the transaction cycle and summarizes the transaction results, analyzing indicators such as transaction efficiency, social welfare, and return distribution. Cross-validation is used to tune hyperparameters, and gradient descent is used to continuously optimize model parameters to ensure the robustness of the method under noise and uncertainty.

[0012] Optionally, it also includes collecting real-time data to optimize the reinforcement learning model based on transaction data, and updating parameters through online learning, including the following steps: Data collection and storage: During the transaction process, the status, actions, rewards and next state sequence of prosumers and consumers are collected in real time to form a dataset, which is stored based on a distributed database; Parameter update mechanism: The policy network and value network of the multi-agent reinforcement learning algorithm are optimized by combining online learning and batch learning. Online learning processes new data in real time, while batch learning retrains the Markov decision process periodically. Gradient descent-like methods are used to update the parameters of the policy network and value network. Performance monitoring and triggering mechanism: Set thresholds to monitor the performance of the multi-agent reinforcement learning algorithm, and use a sliding window to evaluate data from the most recent trading cycles.

[0013] A microgrid P2P electricity trading bidding and matching optimization system includes: The market modeling module is used to construct a bilateral P2P electricity trading market model for microgrids, defining buyers as energy-deficient microgrids and sellers as energy-rich microgrids, and designing a transaction matching and settlement mechanism based on a second-price auction mechanism. The strategy learning module is used to model the trading environment in the market model as a Markov decision process and run a multi-agent reinforcement learning algorithm under the Markov decision process framework to learn and generate dynamic bidding strategies for multi-round trading. The dynamic bidding strategies include: electricity price, electricity demand and electricity supply. The transaction matching module is used to receive dynamic bidding strategies from producers and consumers during the market matching phase, input them into the microgrid bilateral P2P electricity trading market model, and output a settlement list.

[0014] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0015] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0016] The technical solution provided by the embodiments of the present invention has the following advantages compared with the prior art: This invention provides a microgrid P2P electricity trading bidding and matching optimization method. By constructing a bilateral P2P electricity trading market model involving both energy-deficient and energy-rich microgrids, it introduces a transaction matching and settlement mechanism based on a second-price auction mechanism. This ensures economic characteristics such as incentive compatibility and Nash equilibrium, significantly improving the matching efficiency and resource allocation efficiency of electricity trading under complex supply and demand environments. Compared with traditional electricity trading methods relying on fixed electricity prices, manual experience rules, or static centralized optimization, this invention, while guaranteeing economic characteristics such as incentive compatibility and Nash equilibrium, enables the bids and offers of both buyers and sellers to spontaneously reach a balance between individual benefits and systemic social welfare under the constraints of the mechanism. This helps to curb manipulative bidding behavior and improve the fairness, transparency, and operational economy of the P2P electricity trading market.

[0017] This invention models the P2P electricity trading environment using Markov decision processes. It employs a multi-agent reinforcement learning algorithm to model each microgrid prosumer as an agent, generating an optimal bidding strategy for each. The actions are based on electricity price and demand or supply. Through multiple rounds of trading interactions, the model parameters are continuously optimized and updated using transaction data, thus achieving adaptive adjustment of the bidding strategy. Compared to traditional static or rule-based bidding strategies, this invention maintains good learning and adaptability even under conditions of varying participant numbers, fluctuating electricity prices, and load uncertainty. This allows prosumers to automatically form better electricity purchase and sale decisions in multiple scenarios and operating conditions, improving the robustness and generalization performance of the strategy. Attached Figure Description

[0018] Figure 1 A flowchart of a microgrid P2P power trading bidding and matching optimization method provided in the first part of the embodiments of the present invention; Figure 2 The flowchart illustrates a microgrid P2P power trading bidding and matching optimization method provided in the second part of the embodiments of the present invention. Detailed Implementation

[0019] The following detailed description of a specific embodiment of the present invention is provided in conjunction with the accompanying drawings. However, it should be understood that the scope of protection of the present invention is not limited to the specific embodiment.

[0020] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the technical solution of this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0021] The present invention will be described below through several specific embodiments. To keep the following description of the embodiments clear and concise, detailed descriptions of known functions and components may be omitted. When any component of an embodiment of the present invention appears in more than one drawing, the component may be represented by the same reference numerals in each drawing.

[0022] like Figure 1 As shown, the first aspect of this invention provides a microgrid P2P electricity trading bidding and matching optimization method, comprising the following steps: A bilateral P2P electricity trading market model for microgrids is constructed, defining the buyer prosumer as an energy-deficient microgrid and the seller prosumer as an energy-rich microgrid, and designing a transaction matching and settlement mechanism based on a second-price auction mechanism; The trading environment in the market model is modeled as a Markov decision process. A multi-agent reinforcement learning algorithm is introduced under the Markov decision process framework to train strategies for each producer and consumer, learn and generate dynamic bidding strategies for multiple rounds of trading. The dynamic bidding strategies include: electricity price, electricity demand and electricity supply. The reward function is based on individual utility. Through centralized training and distributed execution framework, the agents learn the optimal strategy, realize Nash equilibrium and maximize social welfare, and improve the bidding adaptation capability. In the market matching phase, by inputting the dynamic bidding strategies of prosumers into the microgrid bilateral P2P electricity trading market model and outputting a settlement list, the microgrid P2P electricity trading bidding and matching optimization method is realized, ensuring incentive compatibility and budget balance. The computational complexity is reduced through optimization algorithms, adapting to real-time decision-making in large-scale markets.

[0023] This invention provides a microgrid P2P electricity trading bidding and matching optimization method. By constructing a bilateral P2P electricity trading market model involving both energy-deficient and energy-rich microgrids, it introduces a transaction matching and settlement mechanism based on a second-price auction mechanism. This ensures economic characteristics such as incentive compatibility and Nash equilibrium, significantly improving the matching efficiency and resource allocation efficiency of electricity trading under complex supply and demand environments. Compared with traditional electricity trading methods relying on fixed electricity prices, manual experience rules, or static centralized optimization, this invention, while guaranteeing economic characteristics such as incentive compatibility and Nash equilibrium, enables the bids and offers of both buyers and sellers to spontaneously reach a balance between individual benefits and systemic social welfare under the constraints of the mechanism. This helps to curb manipulative bidding behavior and improve the fairness, transparency, and operational economy of the P2P electricity trading market.

[0024] This invention models the P2P electricity trading environment using Markov decision processes. It employs a multi-agent reinforcement learning algorithm to model each microgrid prosumer as an agent, generating an optimal bidding strategy for each. The actions are based on electricity price and demand or supply. Through multiple rounds of trading interactions, the model parameters are continuously optimized and updated using transaction data, thus achieving adaptive adjustment of the bidding strategy. Compared to traditional static or rule-based bidding strategies, this invention maintains good learning and adaptability even under conditions of varying participant numbers, fluctuating electricity prices, and load uncertainty. This allows prosumers to automatically form better electricity purchase and sale decisions in multiple scenarios and operating conditions, improving the robustness and generalization performance of the strategy.

[0025] Optionally, constructing a microgrid bilateral P2P electricity trading market model includes the following steps: First, a bilateral P2P electricity trading market model for microgrids is constructed. Electricity producers and consumers participating in the market are defined as microgrid units. Microgrids with a positive electricity deficit, requiring external electricity purchases, are defined as energy-deficient microgrids, acting as buyers; microgrids with a surplus of electricity, capable of selling it externally, are defined as energy-rich microgrids, acting as sellers.

[0026] Based on the above definition and using a distributed architecture, each prosumer independently submits bidding information to the market platform through a proxy node. This bidding information includes: electricity price quotes. (RMB / kWh) and corresponding electricity consumption (kWh), where buyers (prosumers) submit their electricity demand and sellers (prosumers) submit their electricity supply. The market platform aggregates all bidding information within each discrete trading cycle, forming a bilateral P2P electricity trading market.

[0027] To ensure market fairness and economy, this invention, within the aforementioned market framework, designs a transaction matching and settlement mechanism based on a second-price auction mechanism. With the overall goal of maximizing social welfare, it optimizes the allocation of electricity transactions between buyers (prosumers) and sellers (prosumers). This second-price auction-based transaction matching and settlement mechanism considers the utility of buyers (prosumers), the revenue of sellers (prosumers), and the overall welfare of the system when determining the transaction price and volume, satisfying economic characteristics such as incentive compatibility and budget balance. Specifically, it includes:

[0028] The objective function for maximizing social welfare in the market model is: , Constraints: , in, To gather buyers and consumers, A collection of sellers and consumers. Indicates buyer and consumer With sellers and consumers The amount of electricity traded. The price quoted by the buyer and seller. The price quoted by the seller to the consumer. This indicates the upper limit of supply for sellers and consumers. This indicates the maximum demand from buyers and consumers.

[0029] By employing the aforementioned objective function for maximizing social welfare and its constraints, this study optimizes the allocation of electricity transactions between buyer and seller prosumers. Under the premise of satisfying the power supply capacity and electricity demand constraints of each microgrid, it aims to maximize the system's social welfare, providing a unified optimization objective for subsequent transaction matching and bidding strategy learning. In simpler terms, given the bids from buyer and seller prosumers and physical constraints, it seeks an optimal allocation of electricity transactions to maximize the overall social welfare of the system.

[0030] Considering the uncertainty of microgrid power generation and consumption behavior due to factors such as weather and load fluctuations, and the randomness and volatility of producer-consumer (P2P) bidding behavior, this invention introduces a probabilistic model to predict P2P behavior in market modeling. A generative adversarial network (GAN) is trained using historical transaction data, including bidding information submitted by each P2P participant across multiple historical trading periods and their corresponding transaction results. The bidding information includes electricity prices and electricity demand or supply, used to simulate the distribution of P2P bids and improve the robustness of the market model under uncertain environments. Meanwhile, when the number of participating P2P participants is large, direct optimization of the overall market leads to a rapid increase in computational complexity and makes real-time decision-making difficult. Therefore, this invention adopts a hierarchical market structure, dividing the large-scale market into several geographically or electrically similar sub-communities. A clustering algorithm is used to group P2P participants, enabling local P2P transactions within each sub-community, thereby reducing decision-making dimensionality and computational complexity.

[0031] In mathematics, clustering algorithms are used to group producers and consumers, with the objective function being to minimize the within-group variance: , in, x It is a single data point in the dataset. k It refers to the number of community groups. For the community group, The center of mass.

[0032] Reinforcement learning, especially multi-agent reinforcement learning methods, provides a new technical means to solve policy optimization problems in complex dynamic game environments. Existing research has attempted to use reinforcement learning for policy learning in electricity market clearing, demand response, and microgrid dispatching. However, several prominent problems remain in P2P electricity trading scenarios: First, some methods employ single-agent or centralized learning frameworks, failing to fully characterize the strategic interactions between multiple producers and consumers; second, the construction of reinforcement learning environments is often decoupled from specific market mechanisms, making it difficult to guarantee that the learned policies satisfy key economic characteristics; third, when the number of participants is large and the dimensions of the state and action spaces are high, traditional reinforcement learning methods are insufficient in terms of convergence speed and stability, making it difficult to meet the requirements of engineering applications.

[0033] To address the aforementioned issues, this invention, based on a pre-constructed P2P electricity trading market, employs a multi-agent reinforcement learning algorithm to generate dynamic bidding strategies for each prosumer. This invention models each microgrid prosumer as an agent and the electricity trading process as a Markov Decision Process (MDP). Within the Markov Decision Process framework, a multi-agent reinforcement learning algorithm is introduced to learn and generate dynamic bidding strategies for multiple rounds of trading, enabling the agents to learn optimal bidding behavior through repeated market interactions. Specifically, the process includes the following steps:

[0034] (0) MDP environment modeling based on the second-price auction mechanism (McAfee mechanism): First, the P2P electricity trading market based on the McAfee mechanism is formalized into an MDP environment.

[0035] To approximate the Markov property in a finite-dimensional state and enable the agent to simultaneously perceive price, supply and demand, and competitive landscape information, the state space design in this embodiment of the invention is as follows: [States...] Including overall market information and individual characteristics, formalized as follows: , in, Historical electricity prices For supply forecasting, For demand forecasting, This represents the proportion of buyers. The proportion of sellers' quantities. This represents the distribution characteristics of the previous round of bidding. Through the design of the state space, the learned bidding strategy acquires a clear economic meaning and can adaptively adjust to the market environment.

[0036] To ensure that actions directly correspond to the producer-consumer's controllable "price + electricity" decision and to adapt to reinforcement learning algorithms for continuous control, the action space design in this embodiment of the invention is as follows: Each producer-consumer's actions... ,in Price quote (RMB / kWh) The value is in kWh. The action range is constrained by the McAfee mechanism, such as requiring bids to be within a reasonable market range. Action space design allows for reinforcement learning outputs to be directly used as bidding decisions, improving strategy interpretability and training convergence efficiency.

[0037] To accurately characterize the state evolution resulting from the interaction of multiple agents through market mechanisms using actual McAfee auction rules, in this embodiment of the invention, state transition dynamics are defined: environmental state transitions are driven by McAfee rules, and state transition results are generated by clarifying the processing flow of the second-price auction mechanism in each round of transactions, namely: , in, Indicates that the current state is known. And perform the action Under the condition of, the next state The probability of survival serves as a bridge connecting the agent's actions with environmental feedback. This is the deterministic transfer function of the McAfee mechanism. This ensures that the final strategy becomes the rational optimal response to the given auction mechanism, and the system's outcome is closer to a high-welfare equilibrium state.

[0038] (2) The core processing flow of the McAfee mechanism: After completing the MDP environment modeling, in order to tightly couple the actual market settlement results with the reinforcement learning process, it is necessary to clarify the core processing flow of the McAfee mechanism in each round of transactions. This flow is used both to generate state transition results and to provide key intermediate variables for the design of the reward function and the input of the critic network (critical price: the unified settlement reference price calculated based on the order of bids from buyers and sellers in the core McAfee process; matching indicator function: a 0 / 1 variable indicating whether a certain agent wins the bid in this round), as detailed below:

[0039] Bid ranking: Buyer and consumer bids are ranked in descending order. Sorting seller and consumer prices in ascending order. ; Key price calculation: Find the maximum index , making The key price is ; Matching and Settlement: Only for and The agent is assigned positive rewards to ensure incentive compatibility.

[0040] This invention modifies the conventional McAfee mechanism by incorporating reinforcement learning. Specifically, it explicitly embeds the traditional McAfee mechanism into a Markov decision process and a multi-agent deep deterministic policy gradient framework. This not only completes the ordering of buyers and sellers, determination of critical transaction pairs and key prices according to conventional steps, but also designs each round of clearing as an environment state transition operator, outputting results such as traded electricity volume, settlement price, individual returns, and social welfare. Simultaneously, intermediate variables such as key price, critical transaction pairs, and winning bid indicators are extracted and used as part of the reward function design and critic network input, constructing a dedicated interface between the McAfee mechanism and multi-agent reinforcement learning. Through this modification, the McAfee mechanism is no longer just an independent settlement module, but is tightly coupled with the reinforcement learning process, enabling the bidding strategies learned by the agents to become rational and optimal responses to given auction rules, balancing individual returns and social welfare improvement. Explicitly utilizing intermediate variables in the mechanism helps improve the accuracy and convergence speed of value estimation and strategy updates, enhances the interpretability and stability of the strategy, and ensures that the final transaction results meet economic characteristics such as budget balance and individual rationality, making it easier for subsequent theoretical analysis and engineering deployment.

[0041] (3) McAfee-MARL ensemble algorithm: Based on the aforementioned environmental modeling and mechanism solving, the McAfee mechanism is integrated with a multi-agent deep reinforcement learning framework to achieve joint learning and updating of agent bidding strategies: Each intelligent agent (i.e., each microgrid producer-consumer) i Individual reward function design: reward Integrating individual utility with mechanism characteristics: , in, Match the indicator function for McAfee (1 for the correct indicator, 0 otherwise). For transactional utility, For cost, Social benefits calculated for McAfee For weights, weights The initial weight was set to 0.5, determined through experimental tuning, to ensure that individual utility and social welfare have equal influence on the reward function. During training, hyperparameters were tuned using grid search to optimize weight values ​​for different market demands and task objectives, ultimately selecting the weights that best balance individual incentives and global social welfare.

[0042] Multi-agent deep reinforcement learning algorithm: Employing the Multi-agent Deep Deterministic Policy Gradient Framework (MADDPG) and critic network. The input includes intermediate variables from the McAfee mechanism output (the critic network is a module within the multi-agent deep deterministic policy gradient framework. As a value function approximator in the multi-agent deep deterministic policy gradient framework, the critic network takes the global state and the joint actions of each agent as input during the centralized training phase, outputs estimates of the rewards for each agent, and provides gradient signals to the policy networks of each agent, thereby achieving joint learning and updating of the multi-agent bidding policy).

[0043] , in, Let be the gradient of the objective function, representing the agent's... i objective function J (i.e., expected cumulative reward) relative to its policy network parameters The gradient direction. Update parameters along this direction. This can maximize its long-term returns. E The expectation operator represents the mathematical expectation of the expression within the parentheses, i.e., considering the agent's adherence to the current policy. The average value across all possible scenarios during interaction with the environment ensures the stability and unbiasedness of parameter updates. Let the gradient of the action-value function represent the centralized commentator network. The output value relative to the agent i action The gradient. It indicates the gradient of the current state s and the joint action. a Next, fine-tuning the movements How this will affect future expected total returns provides "improvement directions" for strategy updates. Let the policy gradient represent the agent's gradient. i policy function Relative to its policy network parameters The gradient. It indicates how to fine-tune the parameters. This will make the strategy more inclined to choose the current action. It adds two key pieces of information from the results of market mechanisms to the traditional input. Key price and The matching indicator function allows the Q function to directly and accurately evaluate the value of a bidding action under real market rules. This is an action feasibility indicator function, when the action... Under the McAfee mechanism, its value is 1 when it is "feasible" or "reasonable", and 0 otherwise.

[0044] Policy gradient update: , Through the above (1)-(3), the complete link from "MDP modeling based on McAfee mechanism, to core mechanism calculation, and finally multi-agent deep reinforcement learning integration and policy update" is completed, providing an initial bidding strategy for subsequent simulation and online optimization.

[0045] Optionally, by inputting the dynamic bidding strategies of prosumers into the microgrid bilateral P2P electricity trading market model, and outputting a settlement list, that is, by applying an auction mechanism to perform market matching and pricing based on the bidding information of prosumers, and outputting the transaction results, the following steps are included: Bid information preprocessing and verification: Collect all bid information submitted by producers and consumers. ,in For the quote, The data is for electricity consumption. The validity of bids is verified, and obvious outliers are removed (such as negative bids or excessive electricity consumption), ensuring that the input data meets the requirements of the auction mechanism.

[0046] Bidding information preprocessing and verification: After passing through the core processing flow of the McAfee mechanism, the transaction results are output: a matching matrix of the transaction volumes for both buyers and sellers is output. ,in Indicates buyer and consumer i With sellers and consumers j The system records the transaction volume and generates a settlement list, which includes the transaction volume, settlement price, and transaction revenue for each producer and consumer.

[0047] This invention incorporates the McAfee bilateral auction mechanism and the requirements of multi-agent reinforcement learning in both the preprocessing of bidding information and the output of settlement results. On the one hand, it eliminates abnormal bids through dual constraints of mechanism feasibility and physical feasibility, ensuring the stability of the transaction and training process. On the other hand, it outputs the matching result matrix and individual individual payoffs, providing refined reward and state transition information for reinforcement learning, thereby achieving closed-loop optimization of the bidding strategy. This is different from the conventional settlement steps that only perform a one-time market clearing.

[0048] This invention provides a microgrid P2P electricity trading bidding and matching optimization method, which further includes simulating and verifying the microgrid bilateral P2P electricity trading market model, dynamic bidding strategy, and transaction matching and settlement mechanism through a simulation platform. The simulation platform verifies the trading mechanism and bidding strategy, and the P2P electricity trading mechanism is solved, including the following steps: To verify the effectiveness and feasibility of the aforementioned trading mechanism and bidding strategy, this invention simulates the method using a P2P electricity trading simulation platform. Specifically, on the one hand, the electricity consumption or generation status of each microgrid prosumer and its external environment information is simulated and generated through a prosumer agent server; on the other hand, based on a multi-agent reinforcement learning algorithm, a corresponding dynamic bidding strategy (including electricity price and quantity) is generated for each prosumer and sent to the local server.

[0049] On a local server, using MATLAB / Simulink, the transaction matching and settlement mechanism based on the second-price auction mechanism is solved: input the dynamic bidding strategy of producers and consumers, complete one or more transaction processes according to the matching and pricing rules, and obtain the transaction results of each round of transactions.

[0050] The simulation platform continuously simulates multiple transaction cycles, summarizes the transaction results of each producer and consumer in different scenarios and time periods, analyzes transaction efficiency, social welfare, and revenue distribution indicators, and displays them in a visual way, providing an intuitive basis for evaluating bidding strategies and transaction mechanisms.

[0051] This invention utilizes a simulation platform to calculate and verify the proposed trading mechanism and bidding strategy. It integrates the market model, auction rules, and reinforcement learning strategy into a unified simulation environment, quantitatively evaluating trading efficiency, social welfare, and strategy stability. This allows the proposed method to undergo mechanistic analysis and performance verification before engineering application. Compared to traditional trading design methods that struggle to systematically evaluate effectiveness before operation, this invention not only optimizes the electricity purchase and sale behavior of a single microgrid but also possesses good scalability. It can be applied to P2P electricity trading and energy internet scenarios between multi-regional microgrids, helping to reduce the overall cost of electricity trading and dispatch, improve renewable energy consumption levels, and enhance the overall safety and stability of the microgrid system.

[0052] The actual behavior of microgrid prosumers is influenced by a variety of uncertainties, such as fluctuations in renewable energy output, expected changes in electricity prices, user comfort preferences, and risk appetite. This results in significant time-varying and heterogeneous optimal bidding strategies. Existing methods often struggle to simultaneously maximize individual benefits and improve overall social welfare within a unified framework. They also lack mechanisms for closed-loop optimization and continuous learning using historical transaction data, leading to strategies that are prone to degradation or deviation from the market optimum in the long run.

[0053] To address the aforementioned problems, this invention provides a microgrid P2P electricity trading bidding and matching optimization method, which further includes collecting real-time data based on a reinforcement learning model optimized from transaction data, updating parameters through online learning, and achieving adaptive updating of the bidding strategy, comprising the following steps: Data collection and storage: During the transaction process, the states, actions, rewards, and next state sequences of prosumers and consumers are collected in real time to form a dataset. Data storage is based on distributed databases (such as blockchain or cloud platforms) to ensure integrity and traceability. For example, state Including variables such as electricity prices, supply and demand, actions For bid pricing, a reward Calculated based on individual utility;

[0054] Parameter Update Mechanism: Based on the constructed dataset, this invention employs a combination of online learning and batch learning to directly optimize the parameters of the policy network and value network of each agent in the multi-agent reinforcement learning algorithm, achieving continuous improvement of the bidding strategy. The reinforcement learning model parameters are optimized using a combination of online and batch learning. Online learning processes new data in real time to adapt to market changes; batch learning periodically retrains the model to prevent drift. Key Algorithms (parameter update algorithms using gradient descent methods (such as SGD / Adam) to update the policy network and value network parameters within this multi-agent deep deterministic policy gradient framework (MADDPG class)) include gradient descent for updating policy network parameters. Sum value network parameters The learning rate is dynamically adjusted to balance convergence speed and stability.

[0055] Performance Monitoring and Triggering Mechanism: After establishing the parameter update strategy, this invention further constructs a performance monitoring and triggering mechanism. The update process in this mechanism is no longer blindly executed at fixed intervals, but rather adaptively triggered based on feedback from real transaction data, thus forming a closed-loop control. Thresholds are set to monitor model performance, such as when the average reward... Retraining is triggered when the decline exceeds 10% or social welfare fluctuations exceed 5%. Simultaneously, a sliding window is used to evaluate data from the most recent 100 trading cycles to ensure timely optimization. This 100-cycle dataset maintains the model's original performance while allowing for real-time monitoring based on recent data, preventing data drift.

[0056] This invention achieves parameter adaptation through closed-loop feedback, ensuring the bidding strategy remains highly efficient over long-term operation. The parameter optimization process is based on mathematical formulas to ensure rigor; key formulas include:

[0057] Stochastic Gradient Descent (SGD) update rule: used for online learning, parameters The update formula is: , in, For learning rate, This is the gradient of the loss function.

[0058] Loss function: Evaluates the deviation between the predicted value and the target value, used in the supervised learning phase. , in, For the target Q value, This is the current network prediction value. N For batch size. Adaptive learning rate adjustment: based on a time decay strategy to prevent oscillations, the formula is:

[0059] , in, The initial learning rate, The attenuation coefficient is... t For training rounds.

[0060] refer to Figure 2 The second aspect of this invention provides a microgrid P2P electricity trading bidding and matching optimization system, comprising: The data preparation module collects multi-source data related to microgrid P2P electricity trading, including electricity quotes from producers and consumers, supply and demand, historical prices, weather conditions, and load fluctuations, and performs data preprocessing. Preprocessing steps include missing value imputation, outlier removal, and data normalization to ensure the consistency and quality of input data, providing standardized input for reinforcement learning models. This module integrates blockchain technology to ensure data security and traceability, adapting to the real-time scheduling needs of smart grids.

[0061] The market modeling module is used to construct a bilateral P2P electricity trading market model for microgrids. It defines the buyer prosumer as the energy-deficient microgrid and the seller prosumer as the energy-rich microgrid, and designs a transaction matching and settlement mechanism based on the second-price auction mechanism.

[0062] The strategy learning module is used to model the trading environment in the market model as a Markov decision process, and run a multi-agent reinforcement learning algorithm within the Markov decision process framework to learn and generate dynamic bidding strategies for multi-round trading. The dynamic bidding strategies include: electricity price, electricity demand, and electricity supply.

[0063] The transaction matching module is used to receive dynamic bidding strategies from producers and consumers during the market matching phase, input them into the microgrid bilateral P2P electricity trading market model, and output a settlement list.

[0064] The simulation verification module is used to simulate and verify the microgrid bilateral P2P electricity trading market model, dynamic bidding strategy and transaction matching and settlement mechanism through the simulation platform, so as to realize market clearing and result calculation in the P2P electricity trading process. The adaptive update module is used to optimize the reinforcement learning model based on transaction data, collect real-time data, and update parameters through online learning to achieve adaptive updates of the bidding strategy.

[0065] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method.

[0066] A fourth aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0067] This invention provides a microgrid P2P electricity trading bidding and matching optimization method. By constructing a bilateral P2P electricity trading market model including prosumers and consumers in energy-deficient and energy-rich microgrids, it introduces matching and settlement rules based on the McAfee second-price auction mechanism. A multi-agent reinforcement learning framework is employed to learn dynamic bidding strategies for each prosumer and consumer. The trading mechanism is solved and the strategy closed-loop optimization is implemented on a simulation platform. This achieves synergistic optimization of individual benefits and system social welfare in complex and uncertain environments, improving the efficiency and intelligence level of microgrid P2P electricity trading. Addressing the problems in existing microgrid P2P electricity trading technologies, such as reliance on experience in bidding strategy design, low trading matching efficiency, difficulty in balancing individual benefits and system social welfare, and insufficient adaptability in multi-agent uncertain environments, this invention provides a microgrid P2P electricity trading bidding and matching optimization method based on multi-agent reinforcement learning. It aims to improve the matching efficiency and social welfare level of electricity trading while ensuring market fairness and incentive compatibility, and enhance the adaptability and robustness of trading strategies in complex dynamic scenarios.

[0068] The above inventions are merely a few specific embodiments of the present invention. However, the embodiments of the present invention are not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A microgrid P2P electricity trading bidding and matching optimization method, characterized in that, Includes the following steps: A bilateral P2P electricity trading market model for microgrids is constructed, defining the buyer prosumer as an energy-deficient microgrid and the seller prosumer as an energy-rich microgrid, and designing a transaction matching and settlement mechanism based on a second-price auction mechanism; The trading environment in the market model is modeled as a Markov decision process. A multi-agent reinforcement learning algorithm is introduced under the Markov decision process framework to learn and generate a dynamic bidding strategy for multiple rounds of trading. The dynamic bidding strategy includes: electricity price, electricity demand and electricity supply. During the market matching phase, the dynamic bidding strategy of producers and consumers is input into the microgrid bilateral P2P power trading market model, and a settlement list is output.

2. The microgrid P2P electricity trading bidding and matching optimization method as described in claim 1, characterized in that, The construction of the microgrid bilateral P2P electricity trading market model includes the following steps: Microgrid units are defined as electricity producers and consumers participating in the market. Among them, microgrids with a positive electricity deficit that need to purchase electricity from external sources are defined as energy-deficient microgrid units, which are buyers and consumers. Microgrid units with a surplus of electricity that can sell electricity to external sources are defined as energy-rich microgrids, which are sellers and consumers. Based on the above definition and a distributed architecture, each producer and consumer independently submits bidding information, including electricity price and quantity, through a proxy node. The market platform aggregates all bidding information in each discrete trading cycle to form a bilateral P2P electricity trading market.

3. The microgrid P2P electricity trading bidding and matching optimization method as described in claim 2, characterized in that, The design, based on a second-price auction mechanism, includes a transaction matching and settlement mechanism that includes: With the goal of maximizing social welfare, an objective function and its constraints are established to optimize the allocation of electricity transactions between buyers and sellers, thereby maximizing the social welfare of the system while meeting the power supply capacity and electricity demand constraints of each microgrid. In the process of market modeling, a probabilistic model is introduced to predict the behavior of prosumers, and historical data is used to train a generative adversarial network to simulate the bidding distribution. By adopting a hierarchical market structure, the large-scale market is decomposed into multiple sub-communities, and clustering algorithms are used to group prosumers and consumers, enabling local P2P transactions to take place within each sub-community.

4. The microgrid P2P electricity trading bidding and matching optimization method as described in claim 1, characterized in that, The process of modeling the trading environment in the market model as a Markov decision process, and introducing a multi-agent reinforcement learning algorithm within the Markov decision process framework to learn and generate dynamic bidding strategies for multi-round trading includes the following steps: The P2P electricity trading market based on the second-price auction mechanism is formalized as a Markov decision process. The state space contains global market information and individual characteristics, the action space contains the actions of each producer and consumer, and the environmental state transition in the state transition dynamics is driven by the McAfee rule. The state transition results are generated by clarifying the processing flow of the second-price auction mechanism in each round of trading. The second-price auction mechanism is integrated with a multi-agent reinforcement learning algorithm to achieve joint learning and updating of agent bidding strategies. The multi-agent reinforcement learning algorithm adopts a multi-agent deep deterministic policy gradient framework, in which the input of the critic network includes the state transition results generated by the second-price auction mechanism.

5. The microgrid P2P electricity trading bidding and matching optimization method as described in claim 1, characterized in that, The process of inputting the dynamic bidding strategy of prosumers into the microgrid bilateral P2P electricity trading market model and outputting a settlement list includes the following steps: Bidding information preprocessing and verification: Collect all bidding information submitted by producers and consumers, verify the validity of the bids, and remove outliers; After processing through the second-price auction mechanism, the transaction results are output: the matching result matrix of the transaction volume of each buyer and seller prosumer is output, and a settlement list is generated, which includes the transaction volume, settlement price and transaction revenue of each prosumer.

6. The microgrid P2P electricity trading bidding and matching optimization method as described in claim 1, characterized in that, It also includes simulation verification of the microgrid bilateral P2P electricity trading market model, dynamic bidding strategy, and transaction matching and settlement mechanism through a simulation platform, including the following steps: The electricity consumption or electricity generation of prosumers is simulated and generated through the prosumer agent server, and the dynamic bidding strategy of each prosumer is generated according to the multi-agent reinforcement learning algorithm and transmitted to the local server. On a local server, the dynamic bidding strategy is input into the transaction matching and settlement mechanism based on the second-price auction mechanism to complete one or more transaction processes, and the transaction results of each round of transactions are obtained, thereby verifying the efficiency of the transaction mechanism and the dynamic bidding strategy. The simulation platform continuously simulates the transaction cycle and summarizes the transaction results, analyzing indicators such as transaction efficiency, social welfare, and profit distribution.

7. The microgrid P2P electricity trading bidding and matching optimization method as described in claim 1, characterized in that, It also includes collecting real-time data to optimize reinforcement learning models based on transaction data, and updating parameters through online learning, including the following steps: Data collection and storage: During the transaction process, the status, actions, rewards and next state sequence of prosumers and consumers are collected in real time to form a dataset, which is stored based on a distributed database; Parameter update mechanism: The policy network and value network of the multi-agent reinforcement learning algorithm are optimized by combining online learning and batch learning. Online learning processes new data in real time, while batch learning retrains the Markov decision process periodically. Gradient descent-like methods are used to update the parameters of the policy network and value network. Performance monitoring and triggering mechanism: Set thresholds to monitor the performance of the multi-agent reinforcement learning algorithm, and use a sliding window to evaluate data from the most recent trading cycles.

8. A microgrid P2P electricity trading bidding and matching optimization system, characterized in that, include: The market modeling module is used to construct a bilateral P2P electricity trading market model for microgrids, defining buyers as energy-deficient microgrids and sellers as energy-rich microgrids, and designing a transaction matching and settlement mechanism based on a second-price auction mechanism. The strategy learning module is used to model the trading environment in the market model as a Markov decision process, and run a multi-agent reinforcement learning algorithm under the Markov decision process framework to learn and generate dynamic bidding strategies for multi-round trading. The dynamic bidding strategies include: electricity price, electricity demand and electricity supply. The transaction matching module is used to receive the dynamic bidding strategies of producers and consumers during the market matching stage, input them into the microgrid bilateral P2P electricity trading market model, and output a settlement list.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

Images