Novel power system multi-agent market operation simulation method and device

Abstract

The invention discloses a novel electric power system multi-agent market operation simulation method and device.The simulation method is executed in computing equipment and comprises the steps that a simulation instance is constructed based on configuration information provided by a user, the simulation instance comprises at least one agent object and configuration parameters of the agent object, and the configuration parameters of the agent object are calculated according to the configuration parameters of the agent object; any agent object is an independently packaged deep reinforcement learning autonomous decision-making unit, and the configuration parameters indicate attribute parameters of each agent object, a market type needing to be simulated and policy parameters of a simulation area; driving each agent object in the simulation instance through a simulation engine, enabling the agent objects to carry out interaction deduction based on the configuration parameters, and recording an interaction process to generate a simulation result; and analyzing and processing the simulation result by using a preset data analysis method.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to a novel method and apparatus for simulating the operation of a multi-agent market in a power system. Background Technology

[0002] Guided by the strategic goal of "peak carbon and carbon neutrality," my country's energy structure is undergoing profound changes, accelerating its transition to a new power system dominated by a high proportion of renewable energy. This system, centered on new energy sources such as wind and solar power, and relying on flexible resources like energy storage and demand-side response as key supports, aims to build a clean, low-carbon, safe, and efficient modern energy system. However, the inherent intermittency and strong randomness of new energy power generation significantly exacerbate real-time fluctuations on both the supply and demand sides of the power system, posing unprecedented challenges to the system's stable operation and real-time balance.

[0003] Against this backdrop, an efficient and mature electricity market mechanism has become a key link in achieving optimal allocation of energy resources. Since the launch of a new round of power system reform in 2015, my country has been committed to building an electricity trading system with "participation on both the generation and consumption sides and market-based pricing." In particular, the "Guiding Opinions on Accelerating the Construction of a Unified National Electricity Market System" jointly issued by the National Development and Reform Commission and the National Energy Administration in 2022 further clarified the goal of basically establishing a unified national electricity market by 2030 and comprehensively promoting the participation of new energy in market transactions. However, looking at the current development status, the electricity market mechanism as a whole is still in the exploration and improvement stage, facing a series of technical problems that urgently need to be solved: First, the market is fragmented, lacking effective connection and coordination mechanisms between markets of different time scales such as the medium- and long-term market, the spot market, and the ancillary services market; second, the participating entities are incomplete, with insufficient depth and breadth of participation in market transactions by multiple entities such as users, and their behavioral characteristics have not been fully reflected; third, simulation verification methods are lacking, with a lack of effective tools that can accurately simulate the complex game behavior of multiple entities and systematically simulate and verify the impact of market rules and policies.

[0004] Therefore, in order to solve the above-mentioned technical challenges and support the scientific planning and stable operation of the new power system, it is urgent to develop a high-fidelity simulation system and method that can fully simulate the entire process of market operation and accurately depict the adaptive decision-making behavior of multiple stakeholders. This is intended to provide crucial technical support and decision-making basis for verifying the effectiveness of market rules, validating the rationality of mechanism design, and comprehensively evaluating macro policies. Summary of the Invention

[0005] In view of the above problems, the present invention is proposed to provide a novel method and apparatus for simulating the operation of a multi-agent market in a power system, which overcomes or at least partially solves the above problems.

[0006] According to one aspect of the present invention, a novel multi-agent market operation simulation method for power systems is provided, executed in a computing device. The method includes: constructing a simulation instance based on configuration information provided by a user, wherein the simulation instance includes at least one agent object and configuration parameters of the agent object, each agent object being an independently encapsulated deep reinforcement learning autonomous decision-making unit, and the configuration parameters indicating the attribute parameters of each agent object, the market type to be simulated, and the policy parameters of the simulation area; driving each agent object in the simulation instance through a simulation engine to perform interactive inference based on the configuration parameters, and recording the interaction process to generate simulation results; and analyzing and processing the simulation results using a preset data analysis method.

[0007] Optionally, in the simulation method according to the present invention, each intelligent agent object in the simulation instance is driven by a simulation engine to perform interactive inference based on configuration parameters, and the interaction process is recorded to generate simulation results. This includes: initializing each intelligent agent object according to the configuration parameters to set its initial state; determining its behavior policy for each intelligent agent object based on a reinforcement learning decision function and in combination with the initial state; controlling the corresponding intelligent agent object to execute the simulation process based on each behavior policy to obtain the initial simulation results; adjusting the simulation state of the current intelligent agent object according to the initial simulation results using a preset reward mechanism, and using this as the initial state; and repeatedly executing the steps of determining the behavior policy, controlling the simulation process, and adjusting the simulation state until a preset number of iterations is reached.

[0008] Optionally, in the simulation method according to the present invention, constructing a simulation instance based on configuration information provided by the user includes: presenting a graphical configuration interface to the user, the configuration interface including visual identifiers corresponding to various intelligent agent objects; in response to the user's selection of any visual identifier, loading the corresponding target intelligent agent object into the initial simulation instance; receiving configuration parameters for each target intelligent agent input by the user, and obtaining the final simulation case.

[0009] Optionally, in the simulation method according to the present invention, for each agent object, its behavioral strategy is determined based on the reinforcement learning decision function and in combination with the initial state, including: adopting an ε-greedy strategy for action selection, wherein the exploration factor ε decreases linearly with the increase of the number of simulation iterations.

[0010] Optionally, in the simulation method according to the present invention, the graphical configuration interface further presents a market type selection interface, where the market types include medium- and long-term trading simulation, spot trading simulation, and ancillary service trading simulation; and drives each intelligent agent object in the simulation instance through the simulation engine to perform interactive deduction based on configuration parameters, and records the interaction process to generate simulation results, including: in response to the selection operation performed by the user through the market type selection interface, determining one or more target market types to be simulated in the simulation instance; and based on the determined target market type, calling the corresponding market clearing algorithm to drive the intelligent agent objects to perform interactive deduction under the corresponding market rules.

[0011] Optionally, the simulation method according to the present invention further includes: processing the coupled simulation of the medium- and long-term market and the spot market in the simulation process based on a two-layer optimization framework. The two-layer optimization framework includes an upper-layer optimization model and a lower-layer optimization model. The upper-layer optimization model takes the maximization of the long-term profit of each intelligent agent object as the optimization objective, and its decision variables include the electricity volume and price declared in the medium- and long-term market. The lower-layer optimization model takes the maximization of social welfare in the spot market as the optimization objective, and its decision variables include the cleared electricity volume and cleared price in the spot market. Through a constraint mapping mechanism, the medium- and long-term contract electricity volume determined by the upper-layer optimization model is used as a rigid constraint for execution and introduced into the solution process of the lower-layer optimization model to maintain the consistency between the medium- and long-term contract and the spot clearing results, as well as the correctness of the contract settlement.

[0012] Optionally, the simulation method according to the present invention further includes: determining a behavior strategy based on the role type to which the intelligent agent object belongs, wherein the role type includes at least power supply type and demand type.

[0013] Optionally, in the simulation method according to the present invention, determining the behavior strategy based on the role type to which the intelligent agent object belongs includes: if the intelligent agent object is a power supplier, the behavior strategy includes adaptive pricing based on reinforcement learning or recommended pricing based on historical data; if the intelligent agent object is a demander, the behavior strategy includes power purchase decisions based on load demand.

[0014] Optionally, in the simulation method according to the present invention, the preset data analysis method is at least one of statistical hypothesis testing method, regression and correlation analysis method, time series analysis method, and machine learning and data mining method.

[0015] Optionally, in the simulation method according to the present invention, the deep reinforcement learning autonomous decision-making unit resides in one of the following reinforcement learning models: deep Q-network model, multi-agent deep deterministic policy gradient model, multi-agent dual-delay deep deterministic policy gradient model, proximal policy optimization model, and flexible actor-commentator model.

[0016] According to another aspect of the present invention, a novel multi-agent market operation simulation device for power systems is provided, residing in a computing device. The device includes: a construction module, adapted to construct simulation instances based on configuration information provided by a user, wherein the simulation instance includes at least one agent object and configuration parameters of the agent object, each agent object being an independently encapsulated deep reinforcement learning autonomous decision-making unit, and the configuration parameters indicating the attribute parameters of each agent object, the market type to be simulated, and the policy parameters of the simulation area; a simulation module, adapted to drive each agent object in the simulation instance through a simulation engine, enabling it to perform interactive inference based on the configuration parameters, and recording the interaction process to generate simulation results; and an analysis module, adapted to analyze and process the simulation results using a preset data analysis method.

[0017] According to another aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the methods described above.

[0018] According to another aspect of the present invention, a readable storage medium storing program instructions is provided, which, when read and executed by a computing device, causes the computing device to perform the method described above.

[0019] According to the present invention, the intelligent agent object is configured as an independently encapsulated deep reinforcement learning autonomous decision-making unit, which enables it to dynamically adjust its bidding or electricity purchase behavior based on market feedback. Compared with the traditional fixed strategy model, it can simulate the market price evolution law under long-term game and improve the realism of simulation results.

[0020] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0021] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A schematic diagram of a computing device 100 according to an embodiment of the present invention is shown; Figure 2 A flowchart of a novel power system multi-agent market operation simulation method 200 according to an embodiment of the present invention is shown; Figure 3 A schematic diagram of the system architecture design of a novel power system market operation simulation platform according to an embodiment of the present invention is shown; Figure 4 A schematic diagram illustrating a specific simulation function according to an embodiment of the present invention is shown; Figure 5 A schematic diagram of market integration for a novel power system market operation simulation platform according to an embodiment of the present invention is shown; Figure 6 A schematic diagram of a simulation process for an intelligent subject object according to an embodiment of the present invention is shown; Figure 7 A schematic diagram of the structure of a novel power system multi-agent market operation simulation device 300 according to an embodiment of the present invention is shown. Detailed Implementation

[0022] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0023] Currently, simulation studies on the operation mechanism and behavior of the electricity market mainly follow two technical routes: one is the simulation method based on experimental economics, and the other is the simulation method based on multi-agent modeling.

[0024] 1. Experimental Economics Simulation Methods and Their Limitations This method draws on ideas from economics and game theory. By constructing a virtual trading environment, it invites real participants (such as representatives of power companies or researchers) to conduct bidding, auctions and clearing operations under preset rules, thereby observing the operating results and price patterns under different market designs.

[0025] For example, the carbon trading experimental platform constructed by Thurber et al. explored the price formation mechanism of the carbon market by linking carbon prices with electricity prices. Domestic scholars Chen Haoyong et al. used this method to simulate the bidding behavior of power generators in an oligopolistic market and analyzed market equilibrium and collusion risks; Guo Jifang et al. conducted experiments to study the bidding behavior of power generators and anti-market manipulation rules.

[0026] These methods typically involve building market rules and trading interfaces on a computer, setting up a limited number of supply and demand participants who submit information within a given time period. The system then clears out participants according to a pre-set mechanism and outputs indicators such as the clearing price, transaction volume, and social welfare. Through multiple rounds of repeated experiments, the evolution of the market and the effects of institutional adjustments can be observed. Its advantage lies in its ability to directly capture the complexity of human decision-making, making it suitable for basic rule testing and behavioral economics research.

[0027] However, this method has obvious limitations and is difficult to meet the simulation requirements of new power systems: Poor repeatability and scalability: It relies heavily on the real-time decisions of participants, making it difficult to guarantee the complete rationality and consistency of participants' behavior, which makes it difficult to reproduce and verify the experimental conclusions.

[0028] Simulation scale is limited: Due to limitations in manpower and cost, it can usually only simulate a dozen to several dozen entities, making it impossible to build a giant system model that matches the actual market and includes tens of thousands of participants.

[0029] Lack of algorithmic support: Based on human decision-making, it is difficult to deeply integrate with intelligent computing methods such as reinforcement learning and optimization algorithms, and it is impossible to achieve automated policy search and long-term dynamic simulation.

[0030] Insufficient representation of market coupling: Existing platforms mostly focus on a single market type (such as the day-ahead or carbon market) and lack comprehensive simulation capabilities for the coupling mechanisms between markets at multiple time scales, such as medium- and long-term markets, spot markets, and ancillary services.

[0031] In summary, experimental economic simulations are more suitable for studying micro-level behaviors, but they are less capable of supporting high-fidelity simulations of complex market operations in new power systems at the engineering level.

[0032] 2. Multi-agent modeling and simulation methods and their existing problems With the development of computer and artificial intelligence technologies, simulation methods based on multi-agent modeling have become mainstream. This method simulates the multi-party interactions and price formation process in the market by defining intelligent agents with independent attributes and objectives in a computer.

[0033] For example, the AMES system developed by Tesfatsion et al. was the first to introduce multi-agent modeling into the electricity market, using reinforcement learning to simulate the evolution of generator strategies; the platform designed by North et al. includes multiple types of agents such as generators, retailers, and users, and uses a nodal pricing mechanism for clearing; Praca et al. developed a European electricity market simulation platform for evaluating market rules. Domestic research institutions have also developed similar panoramic experimental platforms for the electricity market.

[0034] By defining market participants through object-oriented modeling, establishing pricing and clearing algorithms based on market rules, simulating transactions at different time scales, and outputting indicators such as price, returns, and stability, this method offers significant improvements in simulation scale, flexibility, and automation compared to experimental economics methods.

[0035] Despite the widespread adoption of this method, the following key issues remain in addressing the requirements of new power systems: Insufficient intelligence level of the subject: In most systems, the subject's decision-making relies on preset fixed rules or simple heuristic strategies, lacking the ability to learn autonomously and optimize strategies in dynamic market environments, resulting in poor adaptability.

[0036] The market structure modeling is too simplistic: the simulation scope is mostly limited to the spot market for electricity, and there is a lack of linkage modeling for the multi-layered market system, including medium- and long-term, retail, and ancillary services, which fails to depict the complex feedback effects between markets.

[0037] The application of advanced algorithms is limited: Although reinforcement learning has been introduced, it is mostly limited to traditional algorithms such as Q-learning. For complex market environments with high dimensions, nonlinearity, and strong temporal sequence, its ability to model the state and action space is insufficient.

[0038] The policy simulation function is weak: existing systems focus more on price and benefit analysis and lack systematic mechanism verification and policy evaluation functions for market designers, making it difficult to quantify the long-term impact of carbon policies and high proportion of new energy access on the market.

[0039] The model lacks user-side and green behavior modeling: the model focuses too much on competition on the generation side and fails to fully integrate demand-side response, dynamic energy storage strategies and user green energy use behavior, and cannot fully simulate the interactive characteristics of energy flow, information flow and value flow in the new power system.

[0040] To address the problems existing in the prior art, this invention proposes a novel multi-agent market operation simulation method for power systems. This method addresses the shortcomings of existing simulation methods described above by solving the following four key technical problems: (1) Solve the heterogeneity problem of multi-agent modeling and realize a unified intelligent modeling system. Existing simulation systems employ varying modeling methods for different market participants, resulting in poor compatibility and interoperability between models and hindering system expansion. This invention aims to establish a unified object-oriented modeling framework. By introducing the abstract structure of "Agent," it achieves unified encapsulation and expansion of diverse participants such as generators, users, energy storage providers, and aggregators, laying a solid model foundation for complex market simulations.

[0041] (2) Break down market hierarchy barriers and realize multi-timescale market linkage simulation. Most existing simulation systems are limited to a single market (such as the spot market) and cannot depict the dynamic coupling relationships and price transmission mechanisms between different markets, such as medium- and long-term markets, spot markets, and ancillary services. This invention aims to construct a multi-timescale collaborative clearing mechanism that organically integrates medium- and long-term transactions, spot bidding, and retail settlement within a unified simulation framework, realistically reproducing the linkage processes and feedback effects between different market levels.

[0042] (3) Enhance the intelligence and adaptability of the subject's behavior to achieve dynamic game simulation. Traditional systems often rely on static strategies or simple rules to drive agent behavior, lacking the ability to learn, adapt, and optimize in complex market environments. This invention aims to introduce deep reinforcement learning algorithms (such as DQN, MADDPG, and MATD3) to enable agents to make autonomous decisions involving state awareness, strategy iteration, and long-term payoff optimization, thereby achieving high-fidelity simulation of the dynamic game-playing behavior of market participants.

[0043] (4) To bridge the gap between theoretical models and engineering applications and provide an end-to-end simulation implementation solution. Existing research largely focuses on theoretical models, lacking a complete, stable, and practically executable software system. This invention aims to design an end-to-end software implementation framework. By constructing a layered architecture of data layer, encapsulation layer, and interface layer, and adopting a client-server collaborative operating mode, it addresses the issues of high concurrency, scalability, and practicality in simulation systems.

[0044] The novel multi-agent market operation simulation method for power systems can be executed in computing device 100. Computing device 100 can be implemented as a server, such as an application server or web server; it can also be implemented as a desktop computer, laptop computer, processor chip, tablet computer, etc., but is not limited to these. Figure 1 A block diagram of the physical components (i.e., hardware) of a computing device 100 is shown. In a basic configuration, the computing device 100 includes at least one processing unit 102 and system memory 104. According to one aspect, depending on the configuration and type of the computing device, the system memory 104 includes, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memories.

[0045] According to one aspect, system memory 104 includes operating system 105. System memory 104 also includes program module 150. According to one aspect, operating system 105, for example, is adapted to control the operation of computing device 100. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other applications, and are not limited to any particular application or system. Figure 1The basic configuration is illustrated by the components within the dashed lines 108. According to one aspect, the computing device 100 has additional features or functions. For example, according to one aspect, the computing device 100 includes additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 1 The middle part is shown by removable storage device 109 and non-removable storage device 110.

[0046] As stated above, according to one aspect, a program module 150 is stored in system memory 104. According to one aspect, the program module 150 can be implemented as one or more computer program products. This application does not limit the type of computer program product, and may include, for example, email, word processing applications, spreadsheet applications, database applications, slideshow applications, drawing or computer-aided applications, web browsers, etc. In some embodiments according to this application, computer programs / instructions related to the novel power system multi-agent market operation simulation method 200 are encapsulated as a computer program product. When these computer programs / instructions are executed by a processor (i.e., processing unit 102), the novel power system multi-agent market operation simulation method 200 according to this application is implemented.

[0047] According to one aspect, examples can be practiced on circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or a microprocessor. For example, it can be practiced via wherein... Figure 1 Each or many of the components shown can be implemented as an example by integrating a System-on-a-Chip (SOC) on a single integrated circuit. According to one aspect, such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operating via the SOC, the functions described herein can be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 100. Embodiments of the invention can also be implemented using other techniques capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Additionally, embodiments of the invention can be implemented within a general-purpose computer or in any other circuit or system.

[0048] According to one aspect, the computing device 100 may also have one or more input devices 112, such as a keyboard, mouse, pen, voice input device, touch input device, etc. It may also include output devices 114, such as a monitor, speaker, printer, etc. The foregoing devices are examples and other devices may also be used. The computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118, such as printing devices, like printers. Examples of suitable communication connections 116 include, but are not limited to: RF transmitters, receivers, and / or transceiver circuitry; Universal Serial Bus (USB); parallel and / or serial ports.

[0049] As used herein, the term computer-readable medium includes computer storage medium. Computer storage medium can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information (e.g., computer-readable instructions, data structures, or program modules). System memory 104, removable storage device 109, and non-removable storage device 110 are examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital universal disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computer device 100. According to one aspect, any such computer storage medium can be part of computing device 100. Computer storage media does not include carrier waves or other transmitted data signals.

[0050] According to one aspect, a communication medium is implemented by computer-readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and includes any information transmission medium. According to one aspect, the term "modulated data signal" describes a signal having one or more sets of characteristics or altered in a manner that encodes information in the signal. By way of example and not limitation, a communication medium includes wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0051] In an embodiment of the invention, computing device 100 is configured to execute a novel power system multi-agent market operation simulation method 200 according to the invention. Computing device 100 includes one or more processors and one or more readable storage media storing program instructions that, when configured to be executed by the one or more processors, cause the computing device to execute the novel power system multi-agent market operation simulation method 200 according to the embodiment of the invention.

[0052] Figure 2 A flowchart of a novel power system multi-agent market operation simulation method 200 according to an embodiment of the present invention is shown. Method 200 is adapted to be executed in a computing device (e.g., the aforementioned computing device 100).

[0053] like Figure 2 As shown, the purpose of method 200 is to realize a method that can simulate the market price evolution law under long-term game and improve the realism of simulation results. It begins with step 202, in which a simulation instance is constructed based on the configuration information provided by the user. The simulation instance includes at least one agent object and the configuration parameters of the agent object. Each agent object is an independently encapsulated deep reinforcement learning autonomous decision-making unit. The configuration parameters indicate the attribute parameters of each agent object, the market type to be simulated, and the policy parameters of the simulation area.

[0054] In some embodiments, the deep reinforcement learning autonomous decision-making unit may house deep Q-networks (DQN), multi-agent deep deterministic policy gradients (MADDPG), multi-agent dual-delay deep deterministic policy gradients (MATD3), proximal policy optimization (PPO), flexible actor-commentator (SAC), etc.

[0055] In some embodiments, step 202 includes the following sub-steps: First, the computing device presents a graphical configuration interface to the user, the configuration interface containing visual identifiers corresponding to various intelligent agent objects. Then, in response to the user's selection of any visual identifier, the corresponding target intelligent agent object is loaded into the initial simulation instance. Finally, the configuration parameters for each target intelligent agent input by the user are received to obtain the final simulation case.

[0056] The graphical configuration interface is a crucial component of the new power system market operation simulation platform, serving as the core entry point for users to interact with the simulation system. The specific system architecture design of this simulation platform is as follows: Figure 3 As shown. The method 200 described in this embodiment is implemented based on the software and hardware architecture built on this platform.

[0057] The new power system market operation simulation platform mainly consists of three parts: data layer, encapsulation layer, and interface layer.

[0058] The data layer is designed based on MySQL, a scalable, high-performance database management system designed for distributed client / server computing. It integrates seamlessly with Windows NT, providing a transaction-based enterprise-level information management system solution. Based on the MySQL database platform, a database module to support the simulation system is developed. Corresponding tables are designed to store the input and output information of different types of park entities and equipment resources. The database structure is set, and the underlying data support for the simulation system is implemented through a database interface. This enables data connectivity between the database and other modules of the system, meeting the system's requirements for real-time data reading, output, modification, deletion, and storage.

[0059] The encapsulation layer uses Python for its main code, chosen because Python is easy to learn, readable, free, open-source, easy to maintain, and relatively fast. Furthermore, Python supports both procedural and object-oriented programming, making it well-suited for multi-agent modeling methods. As a fully object-oriented language, functions, modules, numbers, and strings are all objects, and Python fully supports inheritance, overloading, derivation, and multiple inheritance, which enhances source code reusability.

[0060] The interface layer utilizes a combination of Python and web development to develop the graphical modeling module. Leveraging the advantages of web design—the ability to quickly create visual, flexible, scalable, and reusable animated objects—it addresses the modeling challenges of complex park systems. A flexible web-based animation framework is used to create complex 2D and 3D animations to meet the interactive needs of the simulation system. Users can use a modular approach to create animations for each object individually, then assemble them into a cohesive whole, and edit and control the animations through a rich application programming interface (API).

[0061] The specific simulation functions of this simulation platform are as follows: Figure 4 As shown.

[0062] First, regarding the data layer.

[0063] To ensure the smooth reading of simulation data and the storage of large amounts of simulation output data, the simulation system uses ADO technology to connect to MySQL as a backend database. A database named MEISS_DB is constructed to provide backend support for data retrieval and storage during the simulation process. MEISS_DB contains multiple tables that store the simulation data for each module, including the input data required for the simulation and the output data of the simulation results.

[0064] Then, for the encapsulation layer.

[0065] The packaging layer targets the main market participants: supply and demand entities, electricity contracts, and system resources. The simulation platform uses Python to encapsulate these into classes and includes built-in transaction functions, as shown in Table 1.

[0066] Through class encapsulation, objects such as market entities, resources, and equipment can be mapped to the simulation system. The energy trading, supply, and consumption decisions of each entity are encapsulated into the Agent of the simulation system, realizing object-oriented simulation operation.

[0067] Next, regarding the interface layer.

[0068] The simulation platform's interface is presented through a web design, allowing users to customize case settings according to their needs. Users can drag and drop predefined content items from the interface into the main screen, input the corresponding parameters, and then click the corresponding simulation analysis button after the simulation is complete to begin analysis. Specific analysis content is shown in Table 2 below.

[0069] Table 2 Finally, the simulation platform also includes a connection layer.

[0070] The connection layer connects the interface layer and the encapsulation layer, and the encapsulation layer and the data layer. Different layers are connected using different methods. In the case of connecting Python to the database, pymssql is used. The pymssql package is the driver program (or DB API) for connecting to a MySQL database in Python; it is the tool that ultimately interacts with the database. After connection, the database can directly read experimental data from Python, and Python can directly input experimental results obtained from the data into the database via pymssql, facilitating operations.

[0071] Pypeline is a plugin designed to link a running webpage design model with a locally installed Python installation. Pypeline allows you to import model parameters set up in a webpage into a database and then use Python to read experimental data for experiments, providing a graphical interface for setting up experiments.

[0072] In this embodiment, the construction of simulation cases is accomplished through an intuitive graphical configuration interface. Its core lies in transforming abstract simulation elements (agents, resources, markets) into interactive, visual objects, and dynamically assembling them into executable simulation cases through user drag-and-drop and parameter configuration. The specific process is as follows: 1. Presenting visual icons and initial simulation examples The computing device presents a graphical configuration interface (or "canvas") to the user. This interface clearly displays visual icons or modules representing different types of intelligent agents, such as: a "wind turbine" icon to represent a renewable energy generator, a "factory" icon to represent an industrial power user, a "battery" icon to represent an energy storage operator, and a "cluster" icon to represent a load aggregator, etc.

[0073] Meanwhile, the interface provides a blank initial simulation instance area as a workspace for users to build specific simulation scenarios.

[0074] 2. Load intelligent agent objects via drag-and-drop interaction The simulation platform responds to user drag-and-drop operations on icons. When a user drags and drops a "wind turbine" icon from the toolbar to the simulation instance area, the operation is recognized by the system as a command, which then triggers the following background process: First, the system creates an instance of an agent object of that type in memory based on the agent type corresponding to the icon (such as the ASply supply agent).

[0075] Then, load this newly created agent object and associate it with the current simulation instance.

[0076] Finally, within the simulation instance area, a corresponding graphical representation of the agent is generated that can be selected subsequently.

[0077] 3. Configure parameters to generate the final simulation case. Once all necessary agents have been loaded into the simulation instance, the user can open a configuration dialog box for a specific agent by clicking or double-clicking its graphical representation. The system receives a series of configuration parameters entered by the user in this dialog box. These parameters specifically define the agent's attributes and behaviors, and are typically categorized as follows: Identity and resource parameters: such as name, number, associated generator set capacity, rated power and capacity of energy storage equipment, etc.

[0078] Economic parameters: such as power generation cost, upper / lower price ceiling / floor, and risk appetite coefficient.

[0079] Policy parameters: such as the type of reinforcement learning algorithm used, the initial exploration rate, etc.

[0080] After the user completes the parameter configuration for all intelligent agent objects, the system integrates all information to generate a final simulation case with a complete structure and comprehensive data. This case file contains all the initialization data required for simulation execution and can be directly submitted to the backend simulation engine for execution.

[0081] In a specific example, a campus-level microgrid market is simulated, which includes new energy sources, energy storage, and traditional loads.

[0082] User operation flow: From the left toolbar, the user drags a "Wind Turbine" icon, a "Battery" icon, and a "Factory" icon to the simulation instance area on the right. At this time, the system background has silently created three intelligent agent objects: an ASply (supply agent), an APsmr (energy storage agent), and an ADmd (demand agent).

[0083] Users click on "Wind Turbine Units" within the area and set its parameters in the pop-up dialog box: Name = "Park Wind Power", Rated Capacity = 2MW, Predicted Output Curve = [Import daily wind speed data from file], Pricing Strategy = "Reinforcement Learning (DQN)".

[0084] Users click the "Battery" icon to set its parameters: Name = Enjoy Energy Storage Station, Rated Power = 1MW, Energy Storage Capacity = 4MWh, Charge / Discharge Efficiency = 92%, Operation Strategy = "Peak-Valley Arbitrage".

[0085] Users click the "Factory" icon and set its parameters: Name = Precision Manufacturing Plant, Baseline Load = 1.5MW, Load Curve = [Generated according to production plan], Price Elasticity Coefficient = 0.3.

[0086] The user clicks the "Generate Case" button on the interface. The system packages all the above configuration information (including agent type, relationship, and parameters) into a structured data file, namely the "final simulation case". Then, the user clicks "Start Simulation", and the case is sent to the simulation engine. The engine will drive three agents to conduct games, transactions, and settlements in a virtual electricity market for one day (or longer) based on this case.

[0087] In this way, this embodiment transforms the complex simulation modeling process into an intuitive and efficient graphical configuration operation, greatly reducing the user's learning curve and ensuring the standardization and accuracy of the case construction.

[0088] In some embodiments, the graphical configuration interface also presents a market type selection interface, where market types include medium- and long-term trading simulation, spot trading simulation, and ancillary service trading simulation. In response to a user's selection operation via the market type selection interface, the simulation platform first determines one or more target market types to be simulated in the simulation instance, and then, based on the determined target market type, invokes the corresponding market clearing algorithm to drive the intelligent agent object to perform interactive deduction under the corresponding market rules.

[0089] In a specific example Figure 5 A schematic diagram illustrating market integration of a novel power system market operation simulation platform according to an embodiment of the present invention is shown. Figure 5As shown, the integrated market mainly comprises three parts: medium- and long-term transactions, spot transactions, and ancillary service transactions. In day-ahead spot transactions, the grid dispatch system prepares necessary data before the transaction, including determining unit parameters, operating boundary conditions, and grid operating boundary conditions or constraints. Participating supply and demand entities submit their power supply / demand for the following day and provide corresponding quotes. The dispatching agency matches transactions based on the market participants' submissions using the trading system, determining the transaction volume and price for each time period and clearing the market. Market participants then sign power trading contracts based on the clearing results. In the real-time spot node transaction process, this system uses a 15-minute simulation step, conducting 96 intraday node transactions daily. Supply and demand entities submit their power supply / demand for each intraday node and provide corresponding quotes. After unified clearing by the dispatching agency, real-time node spot contracts are formed. In ancillary service transactions, the simulation platform mainly includes three types of ancillary services provided by supply entities: peak shaving, frequency regulation, and reserve. Ancillary service transactions are divided into day-ahead ancillary service transactions and real-time node ancillary service transactions based on the time scale. In the ancillary service day-ahead transaction simulation process, the participating entities are first identified, including supply entities, demand entities, and energy storage power stations. The types of ancillary services are determined, such as frequency regulation, peak shaving, and emergency power support. Transaction rules and pricing mechanisms are also established, including the range of bid prices, the determination of clearing prices, and cost sharing. Supply entities and demand entities submit applications to the dispatching agency to participate in ancillary services. These applications include information such as unit peak shaving power / load response power and bid prices. Based on the information submitted by each entity, the dispatching agency determines the minimum demand for each ancillary service, calculates the clearing price using an algorithm, and determines the ancillary services that each participating entity needs to provide according to the clearing price and quantity. Finally, ancillary services are prioritized according to the principle of low to high bids, generating an ancillary service day-ahead contract. In the ancillary service real-time node transaction simulation process, supply entities and demand entities submit ancillary service applications for each hourly node to the dispatching agency, generating ancillary service real-time node contracts. The contract generation method is consistent with the ancillary service day-ahead transaction simulation process.

[0090] After the transaction process is completed, the supply and demand entities execute the corresponding energy contracts, implementing real-time energy balance through supply and demand interaction. The real-time balance simulation process uses a 15-minute simulation step and includes two parts: energy supply / consumption and peak shaving, frequency regulation, and backup ancillary services. In the energy supply / consumption simulation process, the supply and demand entities implement energy supply and consumption activities according to the provisions of the energy contracts. This process mainly falls into two categories: first, the supply and demand entities fully implement energy supply and consumption activities in accordance with the energy contracts; second, deviations occur during the execution process. In the second case, the system's supply and demand are unbalanced, requiring other supply entities to provide ancillary services to ensure real-time energy supply and demand balance in the system. When the system's energy supply and demand are unbalanced, the supply entities that have applied for ancillary services adjust their unit output according to the priority of their bids, ensuring real-time energy supply and demand balance from the supply side.

[0091] Subsequently, in step 204, the simulation engine drives each intelligent agent object in the simulation instance to perform interactive inference based on configuration parameters, and records the interaction process to generate simulation results.

[0092] In some embodiments, step 204 includes the following sub-steps: First, each agent object is initialized according to the configuration parameters to set its initial state.

[0093] Then, for each agent, its behavioral strategy is determined based on a reinforcement learning decision function and the initial state. The behavioral strategy can be determined according to the role type to which the agent belongs, with role types including at least power supply type and demand type. Specifically, if the agent is a power supplier, the behavioral strategy includes adaptive pricing based on reinforcement learning or recommended pricing based on historical data; if the agent is a demander, the behavioral strategy includes power purchase decisions based on load demand.

[0094] In some embodiments, an ε-greedy strategy may be used for action selection, wherein the exploration factor ε decreases linearly with the number of simulation iterations.

[0095] Subsequently, the simulation process is executed based on the corresponding intelligent agent objects controlled by each behavioral strategy to obtain the initial simulation results.

[0096] Subsequently, based on the initial simulation results, the simulation state of the current intelligent agent object is adjusted using a preset reward mechanism and used as the initial state.

[0097] Finally, the steps of determining the behavior strategy, controlling the simulation process, and adjusting the simulation state are repeated until the preset number of iterations is reached.

[0098] It should be noted that each agent in the above encapsulation layer has completed model training before encapsulation. The training process is similar to the execution process of step 204, and will not be described in detail here.

[0099] It is easy to understand that the simulation process is the process of solving problems using the deep reinforcement learning model built into the intelligent agent object. In a specific example, taking the DQN algorithm as an example, the simulation process of the intelligent agent object is described in detail below.

[0100] Figure 6 A schematic diagram of a simulation process for an intelligent subject object according to an embodiment of the present invention is shown.

[0101] like Figure 6 As shown, (1) the intelligent subject is initialized, i.e. the evaluation network parameter w is generated. If the current step size step=0, the evaluation network parameter is copied to the target network, the exploration factor ε is generated, the step size α, the discount factor γ, the experience playback batch k, the memory capacity Z and the network update interval N are updated. If the current step size step=0, the current state s of the subject is initialized.

[0102] (2) An ε-greedy strategy is adopted for action selection, i.e., the agent selects a bid. The ε-greedy strategy is widely used in reinforcement learning algorithms to determine the probability of two action modes: randomly exploring the environment and making decisions using a set of strategies. The specific process is as follows: First, an exploration rate ε∈[0,1] is defined. In each bid selection stage, a random number β∈[0,1] is generated. If the random number is less than ε, a random bid is selected within the bid range; otherwise, the optimal bid is selected based on the neural network. The purpose of using the ε-greedy strategy is to enable the agent to fully explore the environment at the beginning of the iteration, thereby accumulating experience in the memory bank as much as possible. As the experience increases, the exploration rate is gradually reduced to increase the possibility of utilizing the learned knowledge. This can balance the needs of exploration and utilization, and continuously optimize the learning and convergence process of the strategy. Therefore, ε is decreased linearly in each training round. In the initial stage, the agent will be more inclined to explore. Then, as learning progresses, the proportion of utilizing known information is gradually increased, i.e.: (1) Where ε t Let ε represent the exploration factor at step t. start ε represents the initial exploration factor. end The exploration factor represents the number of iterations at the end of the simulation, and T is the total number of iterations.

[0103] (3) Execute action a, namely, declare the electricity price. Participating in market bidding is a collaborative layer behavior, because the result of executing action a needs to be reflected in the market clearing result obtained through multi-entity collaboration. Obtain the clearing result according to the market clearing model, and then obtain the next state s' and reward r. Save the information [s,a,r,s'] to the memory bank. If the data in the memory bank reaches the capacity Z, then clear the earliest data put into the memory bank.

[0104] (4) Gradient descent process: The core of the DQN algorithm is to train the parameters w of the neural network to approximate the value function. This paper adopts the gradient descent method to minimize the loss function, that is, the deviation between the target value and the output value of the evaluation network, to continuously train and adjust the network parameters. For state s, the predicted value of the evaluation network output is Q(s,a;w), the reward for performing action a is r, the state transition is s', the target network output is Q(s',a;w'), k samples are selected from the memory bank, and if there are less than k samples, all the information in the memory bank is retrieved. Then the loss function can be expressed as: (2) Next, we calculate the gradient of w and finally update w: (3) (4) (5) Update the environment information, step size step+1. When the step size reaches the network update interval N, copy the evaluation network parameters to the target network.

[0105] (6) Repeat steps (2) to (5) above until step reaches the simulation cycle set for many years. The simulation cycle ends and the simulation ends (or the training ends).

[0106] Finally, in step 206, the simulation results are analyzed and processed using a preset data analysis method. The preset data analysis method is at least one of statistical hypothesis testing, regression and correlation analysis, time series analysis, and machine learning and data mining.

[0107] Based on the extensive data derived from simulations, and applying data analysis methods such as hypothesis testing, the simulation platform can perform various simulation result analyses, such as simulation platform evolution analysis, individual agent behavior process analysis, and multi-action strategy comparative analysis. Simulation platform evolution analysis primarily focuses on the macro-level of the simulation system, specifically simulating the overall process of market transactions in the new power system based on the market mechanisms and policies of the case location. Individual agent behavior process analysis and multi-action strategy comparative analysis, on the other hand, simulate the micro-level, mainly focusing on the trading behavior of various market participants, including the selection of trading parties, bidding decisions, and internal resource allocation in each market.

[0108] The method provided by this invention enables agents to learn their own strategies through a reinforcement learning algorithm (DQN), allowing them to dynamically adjust their bidding behavior based on market feedback. Compared to traditional fixed-strategy models, this invention can simulate the evolution of market prices under long-term game dynamics, improving the realism of simulation results. Through a unified multi-timescale clearing mechanism, it simulates the linkage process of medium- to long-term, spot, and retail markets. Utilizing the hierarchical constraints of the clearing phase reduces redundant calculations and significantly improves simulation efficiency. Modular design and client-server collaboration reduce message transmission and redundant calculations. The server centrally calculates the clearing results, while the client only handles visualization logic, thereby reducing server load by approximately 30% and increasing throughput by approximately 20%. Configurable policy parameters (such as carbon prices, subsidies, and trading rules) allow for rapid evaluation of market outcomes under different policy scenarios, providing decision support for regulatory agencies. Dynamic scheduling of energy storage entities and user-side response simulation can improve the system's green electricity utilization rate. Simulation results can quantify the contribution of policies to carbon emission reduction, assisting in the assessment of "dual carbon" targets.

[0109] Figure 7 A schematic diagram of a novel power system multi-agent market operation simulation device 300 according to an embodiment of the present invention is shown. The device 300 resides in a computing device. The device 300 includes a construction module 302, a simulation module 304, and an analysis module 306 that are coupled to each other.

[0110] The construction module 302 is adapted to construct simulation instances based on user-provided configuration information. Each simulation instance includes at least one agent object and configuration parameters for that agent object. Each agent object is an independently encapsulated deep reinforcement learning autonomous decision-making unit. The configuration parameters indicate the attribute parameters of each agent object, the market type to be simulated, and the policy parameters of the simulation area. The simulation module 304 is adapted to drive each agent object in the simulation instance through a simulation engine, enabling it to perform interactive inference based on the configuration parameters and recording the interaction process to generate simulation results. The analysis module 306 is adapted to analyze and process the simulation results using preset data analysis methods.

[0111] It should be noted that the working principle and process of the device 300 provided in this embodiment are similar to those of the method 200 described above. For relevant details, please refer to the description of the method 200 described above, which will not be repeated here.

[0112] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the present invention.

[0113] A8. The method described in A7, wherein determining the behavioral strategy based on the role type to which the agent object belongs includes: if the agent object is a power supplier, the behavioral strategy includes adaptive pricing based on reinforcement learning or recommended pricing based on historical data; if the agent object is a demand supplier, the behavioral strategy includes electricity purchase decisions based on load demand. A9. The method described in A1, wherein the preset data analysis method is at least one of statistical hypothesis testing, regression and correlation analysis, time series analysis, and machine learning and data mining. A10. The method described in A1, wherein the deep reinforcement learning autonomous decision-making unit resides in one of the following reinforcement learning models: a deep Q-network model, a multi-agent deep deterministic policy gradient model, a multi-agent dual-delay deep deterministic policy gradient model, a proximal policy optimization model, and a flexible executor-commentator model.

[0114] When the program code is executed on a programmable computer, the computing device generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the method of the present invention according to instructions in the program code stored in the memory.

[0115] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0116] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing preferred embodiments of the invention.

[0117] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0118] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in devices as described in this embodiment, or alternatively, can be located in one or more devices different from the device in this example. The modules in the foregoing examples can be combined into a single module or further divided into multiple sub-modules.

[0119] Furthermore, some of the embodiments described herein are methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the methods or method elements forms means for implementing the methods or method elements. Furthermore, the elements described herein in the apparatus embodiments are examples of means for implementing the functions performed by elements for the purposes of carrying out the invention.

[0120] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

[0121] Although the invention has been described with reference to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes, and not for the purpose of interpreting or limiting the subject matter of the invention. Therefore, many modifications and alterations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims.

Claims

1. A novel multi-agent market operation simulation method for power system, executed in a computing device, comprising: constructing a simulation instance based on user-provided configuration information, wherein the simulation instance comprises at least one agent object and configuration parameters of the agent object, any of the agent object being an independently encapsulated deep reinforcement learning autonomous decision unit, and the configuration parameters indicating attribute parameters of each agent object, a market type to be simulated, and policy parameters of a simulation region; driving each agent object in the simulation instance through a simulation engine, so that it interacts and evolves based on the configuration parameters, and records the interaction process to generate a simulation result; analyzing and processing the simulation result using a preset data analysis method.

2. The method of claim 1, wherein, driving each agent object in the simulation instance through a simulation engine, so that it interacts and evolves based on the configuration parameters, and records the interaction process to generate a simulation result, comprising: initializing each agent object according to the configuration parameters to set its initial state; for each agent object, determining its behavior strategy based on a reinforcement learning decision function in combination with the initial state; controlling the corresponding agent object to execute a simulation process based on each behavior strategy to obtain a first-generation simulation result; adjusting the simulation state of the current agent object as the initial state using a preset reward mechanism according to the first-generation simulation result; repeating the steps of determining the behavior strategy, controlling the simulation process, and adjusting the simulation state until a preset number of iterations is reached.

3. The method of claim 2, wherein, constructing a simulation instance based on user-provided configuration information, comprising: presenting a graphical configuration interface to the user, the configuration interface containing visual identifiers corresponding to various types of agent objects; in response to the user's selection of any of the visual identifiers, loading the corresponding target agent object into the initial simulation instance; receiving user input of configuration parameters for each target agent, obtaining the final simulation case.

4. The method of claim 2, wherein, for each agent object, determining its behavior strategy based on a reinforcement learning decision function in combination with the initial state, comprising: using an ε-greedy strategy for action selection, wherein the exploration factor ε linearly decays with the increase of the number of simulation iterations.

5. The method of claim 3, wherein, the graphical configuration interface also presents a market type selection interface, the market type including medium and long-term transaction simulation, spot transaction simulation, and auxiliary service transaction simulation; and driving each agent object in the simulation instance through a simulation engine, so that it interacts and evolves based on the configuration parameters, and records the interaction process to generate a simulation result, comprising: in response to the user's selection operation performed through the market type selection interface, determining one or more target market types to be simulated in the simulation instance; based on the determined target market types, calling corresponding market clearing algorithms to drive the agent objects to interact and evolve under the corresponding market rules.

6. The method of claim 5, wherein, further comprising: The coupling simulation of the medium and long-term market and the spot market in the simulation process is processed based on a double-layer optimization framework, the double-layer optimization framework includes an upper-layer optimization model and a lower-layer optimization model, wherein the upper-layer optimization model takes the long-term profit maximization of each agent object as an optimization target, and the decision variable of the upper-layer optimization model includes the power and price declared in the medium and long-term market, the lower-layer optimization model takes the social welfare maximization of the spot market as an optimization target, and the decision variable of the lower-layer optimization model includes the clearing power and clearing price of the spot market, the medium and long-term contract power determined by the upper-layer optimization model is introduced into the solving process of the lower-layer optimization model as a rigid constraint of execution through a constraint mapping mechanism, so as to keep the consistency of the medium and long-term contract and the spot clearing result and the correctness of the contract settlement.

7. The method of claim 2, wherein, Further comprising: The behavior strategy is determined according to a role type to which the agent object belongs, and the role type at least includes a power supply type and a demand type.

8. A novel multi-agent market operation simulation device of a power system, residing in a computing device, the device comprising: A construction module adapted to construct a simulation instance based on configuration information provided by a user, wherein the simulation instance includes at least one agent object and configuration parameters of the agent object, any agent object is an independently encapsulated deep reinforcement learning autonomous decision unit, and the configuration parameters indicate attribute parameters of each agent object, a market type to be simulated, and policy parameters of a simulation region; A simulation module adapted to drive each agent object in the simulation instance through a simulation engine, so that the agent object interacts and deduces based on the configuration parameters, and records an interaction process to generate a simulation result; An analysis module adapted to analyze and process the simulation result by using a preset data analysis method.

9. A computing device comprising: At least one processor; And A memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, and the program instructions include instructions for executing the method according to any one of claims 1-7.

10. A readable storage medium storing program instructions, when the program instructions are read and executed by a computing device, the computing device executes the method according to any one of claims 1-7.

Images