Multi-agent based power transaction strategy generation method and system

Abstract

The application provides a multi-agent-based power transaction strategy generation method and system, relates to the field of power system strategy optimization generation, and solves the problem of difficulty in balancing transaction revenue robustness and reportable strategy executability. The application obtains a state of charge sequence and accumulated cycle life of each distributed energy storage unit, and a frequency modulation compensation price sequence of a power transaction system; divides the frequency modulation compensation price sequence into time periods to obtain an expected revenue sequence; performs probability density mapping on the state of charge sequence and the accumulated cycle life to obtain a marginal cost function of an agent corresponding to each distributed energy storage unit; takes the marginal cost function as an optimization target to obtain a target allocation scheme; and generates a power transaction strategy including a maximum declared power, a frequency modulation reserved capacity and a declared price of each transaction period based on each target allocation scheme. The application realizes efficient and accurate matching of optimal economic revenue of a power transaction strategy and a power grid frequency modulation demand.

Description

Technical Field

[0001] This application relates to the field of power system automation technology, and in particular to a method and system for generating power trading strategies based on multi-agent systems. Background Technology

[0002] With the deepening of the construction of new power systems, distributed energy storage, as a high-quality, flexible resource for regulation, has shown great application potential in improving grid frequency stability and absorbing fluctuating renewable energy. Participating in the power ancillary services market to obtain frequency regulation compensation revenue has become a key path to promote the commercialization of the distributed energy storage industry and enhance the resilience of grid operation.

[0003] Existing solutions primarily establish economic benefit models for each energy storage entity and utilize distributed algorithms to achieve coordinated allocation of traded electricity. These solutions often focus on following real-time market clearing prices and responding quickly to grid commands, striving to maximize the overall operational efficiency of the system in complex electricity trading environments.

[0004] However, existing solutions lack detailed modeling of the evolution of the internal physical health of energy storage devices when generating trading strategies, and lack effective statistical probability representation of volatile frequency regulation price signals, resulting in poor stability of the generated strategies in actual execution. This not only causes the nonlinear accelerated decay of the energy storage units' lifespan due to frequent frequency regulation, reducing the economics of long-term operation, but also leads to a disconnect between the generated mathematical optimization results and the actual quantity and price declaration specifications in the electricity market. Therefore, existing technologies face the technical challenge of balancing the physical operating lifespan of distributed energy storage with the robustness of trading returns and the executability of declaration strategies under complex market volatility. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for generating power trading strategies based on multi-agent systems, in order to solve the technical problem in the prior art that it is difficult to balance the robustness of trading returns and the feasibility of the application strategy under complex market fluctuations.

[0006] Firstly, this application provides a method for generating power trading strategies based on multi-agent systems, including: Obtain the state-of-charge sequence and cumulative cycle lifetime of each distributed energy storage unit, as well as the frequency regulation compensation price sequence of the power trading system; The frequency modulation compensation price series is divided into time periods to obtain multiple trading periods. The frequency modulation compensation price series in each trading period is then extracted by quantile extraction and kernel density estimation to obtain the expected return series. By performing probability density mapping on the state-of-charge sequence and cumulative cycle lifetime, a margin matrix is ​​obtained. Using a preset lifetime loss function, loss quantification calculation is performed on the expected revenue sequence and margin matrix to obtain the marginal cost function of the agent corresponding to each distributed energy storage unit. Using the marginal cost function as the optimization objective and the margin matrix and the total frequency regulation demand of the power trading system as constraints, the original variables and dual variables are updated and the residuals are corrected through alternating direction multiplier optimization until the preset convergence condition is met, so as to obtain the target allocation scheme including the trading volume and the frequency regulation reserved capacity. Based on each target allocation scheme, the marginal rate of change of each agent's revenue relative to the expected revenue sequence is calculated through multi-agent reinforcement learning nonlinear mapping to obtain the bid price gradient of each agent. The bid price gradient is then discretized to generate a power trading strategy that includes the bid electricity limit, frequency regulation reserved capacity, and bid price for multiple trading periods.

[0007] Optionally, the frequency modulation compensation price series is divided into time periods to obtain multiple trading periods. For each trading period, quantile extraction and kernel density estimation are performed on the frequency modulation compensation price series to obtain the expected return series, including: Based on a preset time duration threshold, the frequency modulation compensation price series is divided into multiple trading periods by time dimension. Based on a preset interval step size, the frequency adjustment compensation price series within each trading period is categorized to obtain multiple price sets; The probability density function of each trading period is obtained by fitting the sampling points falling into each price set using a preset kernel function, and the price mean of each price set is input into the probability density function to obtain the price probability distribution value of each trading period. The expected return for each trading period is obtained by multiplying and summing each price probability distribution value with the mean price in the corresponding price set, and then arranging all the expected returns to obtain the expected return sequence.

[0008] Optionally, a probability density mapping is performed on the charged state sequence and the cumulative cycle lifetime to obtain the margin matrix, including: Based on the preset state division threshold, the range of values ​​for the state of charge sequence and the cumulative cycle life is divided into multiple numerical intervals, and the frequency of occurrence of the state of charge sequence and the cumulative cycle life falling into different numerical intervals is counted to obtain multiple operating state values. The difference between each operating state value and the preset safety boundary threshold is calculated to obtain multiple margin values. The upper and lower limits of power output for each distributed energy storage unit are determined using the margin values. According to the numbering order of the distributed energy storage units, all margin values ​​and their corresponding upper and lower limits of power output are arranged in a matrix to obtain a margin matrix.

[0009] Optionally, using a preset lifetime loss function, loss quantification calculations are performed on the expected revenue sequence and margin matrix to obtain the marginal cost function of the agent corresponding to each distributed energy storage unit, including: By using a preset lifetime loss function, the expected revenue in the expected revenue sequence is numerically mapped to the corresponding margin value in the margin matrix to obtain the loss change rate of each distributed energy storage unit. The value of each loss change rate is converted based on the preset replacement cost to obtain the intermediate loss parameters of each distributed energy storage unit. Based on the product of the margin value of the margin matrix and the rated power of the distributed energy storage unit, the output power range of each distributed energy storage unit is determined, and the first derivative function of each intermediate loss parameter with respect to the output power is constructed by the numerical correspondence of the intermediate loss parameter within the corresponding output power range. The first derivative function is determined as the marginal cost function of the agent corresponding to each distributed energy storage unit.

[0010] Optionally, using the marginal cost function as the optimization objective and the margin matrix and the total frequency regulation demand of the power trading system as constraints, the original and dual variables are updated and residuals are corrected through alternating direction multiplier optimization until the preset convergence condition is met, resulting in a target allocation scheme including traded electricity volume and frequency regulation reserved capacity, including: Set the primary variable to represent the agent's power output value and the dual variable to represent the degree of global imbalance; Within the output power range of each distributed energy storage unit, numerical optimization is performed to search for the desired power value that minimizes the sum of the marginal cost function at the desired power value and the dual variable, which is then used as the updated original variable. The sum of all updated original variables is compared with the total frequency regulation requirement to obtain the residual value; If the residual value is greater than or equal to the convergence threshold, the dual variable is corrected according to the residual value, and numerical optimization is performed within the output power range of each distributed energy storage unit until the residual value is less than the convergence threshold to obtain the target original variable. The margin value of the margin matrix determines the allocation weight of each agent in the total output power. The transaction power and frequency regulation reserved capacity are calculated based on the target original variables and allocation weights. The target allocation scheme is constructed based on the transaction power and frequency regulation reserved capacity.

[0011] Optionally, based on each target allocation scheme, the bid price gradient for each agent is obtained by calculating the marginal rate of change of each agent's revenue relative to the expected revenue sequence, including: Based on the expected returns in the expected return sequence and the transaction electricity in the target allocation scheme, construct the return function corresponding to each agent, and calculate the tangent slope of each agent at the corresponding transaction electricity to obtain the return change value. Each change in revenue is mapped to the corresponding bid price range in the power trading system to obtain multiple bid price gradients; According to the preset power level step size, the power output range corresponding to each intelligent agent is divided into multiple declared power level intervals, and each declared power level interval is determined as a declared power level step.

[0012] Optionally, the bid price gradient is discretized to generate a power trading strategy that includes the bid electricity limit, frequency regulation reserved capacity, and bid price for multiple trading periods, including: Based on each declared price gradient, calculate the price value corresponding to each declared electricity range to obtain the declared price tier for each agent; The transaction volume in each target allocation scheme is used as the upper limit of the declared volume. Based on the time sequence of the transaction period, the upper limit of the declared volume, the declared volume tier, the declared price tier, and the frequency regulation reserved capacity corresponding to each agent are combined to obtain the power trading strategy corresponding to each agent.

[0013] Secondly, this application provides a multi-agent-based power trading strategy generation system, including: The acquisition module is used to acquire the state of charge sequence and cumulative cycle life of each distributed energy storage unit, as well as the frequency regulation compensation price sequence of the power trading system; The segmentation module is used to divide the frequency modulation compensation price series into time periods to obtain multiple trading periods. Then, the frequency modulation compensation price series in each trading period is extracted by quantile extraction and kernel density estimation to obtain the expected return series. The mapping module is used to perform probability density mapping on the state-of-charge sequence and cumulative cycle lifetime to obtain the margin matrix. Using the preset lifetime loss function, the module performs loss quantification calculation on the expected revenue sequence and margin matrix to obtain the marginal cost function of the agent corresponding to each distributed energy storage unit. The update module is used to update the original variables and dual variables and correct the residuals by using the marginal cost function as the optimization objective and the margin matrix and the total frequency regulation demand of the power trading system as constraints, through alternating direction multiplier optimization, until the preset convergence condition is met, and obtain the target allocation scheme including the trading volume and the frequency regulation reserved capacity. The calculation module is used to calculate the marginal rate of change of each agent's revenue relative to the expected revenue sequence based on each target allocation scheme through multi-agent reinforcement learning nonlinear mapping, obtain the bid price gradient of each agent, discretize the bid price gradient, and generate a power trading strategy that includes the bid electricity limit, frequency regulation reserved capacity and bid price for multiple trading periods.

[0014] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute computer programs to implement the steps of the multi-agent-based power trading strategy generation method as described in the first aspect above.

[0015] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the multi-agent-based power trading strategy generation method described in the first aspect above.

[0016] The beneficial effects of this application are: The multi-agent-based power trading strategy generation method provided in this application ensures the comprehensiveness of decision-making by collecting internal physical state data of energy storage units and external market price signals. It solves the problem of poor strategy stability caused by insufficient representation of price signals in existing solutions. It achieves accurate conversion of the internal physical health state of energy storage equipment into the economic cost domain, avoiding nonlinear accelerated degradation of lifespan due to frequent frequency regulation. It achieves efficient convergence of resource allocation among multiple agents, balancing the overall economic efficiency of the power trading system with individual physical constraints while meeting the overall frequency regulation requirements of the power trading system. It also enhances the executability of the submitted strategies.

[0017] Furthermore, this application utilizes a preset lifetime loss function to numerically map each expected return in the expected return sequence to each corresponding margin value in the margin matrix, thereby extracting the loss change rate reflecting the degree of physical loss. Subsequently, based on a preset replacement cost, this change rate is transformed into an economically meaningful intermediate loss parameter. Simultaneously, by combining the margin value with the rated power of the distributed energy storage unit, its output power range is determined. Based on the correspondence between the intermediate loss parameter and the output power range, a first-order derivative function with respect to the output power is constructed. Finally, this function is determined as the marginal cost function of the agent corresponding to each distributed energy storage unit. This solves the technical problems in the prior art, such as reduced operational economics and inaccurate strategy execution caused by the lack of refined physical modeling. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating the multi-agent-based power trading strategy generation method provided in this application embodiment; Figure 2 A flowchart illustrating the method for obtaining the marginal cost function provided in this application embodiment; Figure 3 A flowchart illustrating the method for obtaining a target allocation scheme provided in an embodiment of this application; Figure 4 A schematic diagram of the structure of a multi-agent-based power trading strategy generation system provided in this application embodiment; Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] For the application scenario of distributed energy storage participating in the frequency regulation ancillary service market under the new power system, this application uses kernel density estimation to statistically characterize the volatile frequency regulation price to mitigate revenue risk. Simultaneously, it maps the state of charge and cycle life of the energy storage unit into a dynamic margin matrix, thereby constructing a marginal cost function that can accurately reflect the battery degradation characteristics, thus compensating for the lack of perception of the internal physical health status of energy storage in existing solutions. Subsequently, this application introduces a distributed collaborative optimization mechanism, using an alternating direction multiplier optimization algorithm to seek the optimal solution for power allocation among multiple agents while ensuring privacy. Furthermore, to address the gap between the mathematical optimization results and the power market declaration specifications, gradient discretization is used to transform continuous allocation values ​​into executable quantity and price declaration strategies, ultimately achieving a high degree of balance between the physical operating life of distributed energy storage, the robustness of trading revenue, and the standardization of market declaration.

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The core of this application is to provide a method for generating power trading strategies based on multi-agent systems. A flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes: Step 101: Obtain the state-of-charge sequence and cumulative cycle lifetime of each distributed energy storage unit, as well as the frequency regulation compensation price sequence of the power trading system.

[0023] In this step, a distributed energy storage unit refers to an energy storage device connected to the power distribution network and capable of independently responding to frequency regulation commands. This device may include lithium-ion batteries or lead-carbon batteries. The state-of-charge sequence (SOC) refers to a set of values ​​representing the ratio of the remaining energy of a distributed energy storage unit to its rated capacity, obtained by sampling at fixed time intervals within a preset historical period. This can be represented as... Where k is the energy storage unit number, The total number of sampling time points. Cumulative cycle life refers to the total number of equivalent charge-discharge cycles experienced by a distributed energy storage unit from its initial operation to the current sampling time; it can be used as characteristic data to measure the aging of equipment hardware. The power trading system refers to the price generation environment for submitting applications for power ancillary services and publishing clearing results. The frequency regulation compensation price sequence refers to the set of settlement prices published by the trading environment within a selected time span to compensate distributed energy storage units for participating in frequency regulation services; it can be represented as... .

[0024] In this embodiment, the underlying operational information of each distributed energy storage unit is first obtained through monitoring nodes. Specifically, sensors periodically read the real-time power ratio of the distributed energy storage unit within a selected historical period to obtain a state-of-charge sequence reflecting the dynamic evolution of power. For example, for distributed energy storage units in a specific area, the power percentage sequence over the past 24 hours is obtained. Next, the total throughput of the distributed energy storage unit since its startup is read by accessing the recorded data of the energy management module, and the cumulative cycle life can be obtained after equivalent conversion. Simultaneously, the public database of the trading environment is accessed through the network interface to download the frequency regulation service list price for the selected period and obtain the frequency regulation compensation price sequence.

[0025] For example, in an ancillary service scenario in region A, obtaining distributed energy storage units. Charged state sequence and cumulative cycle life Then, the frequency modulation settlement price per minute for the region over the past week is obtained to obtain the frequency modulation compensation price series. ,in Including a series of price data such as These data provide comprehensive support for subsequent time period segmentation and probability distribution prediction.

[0026] Step 102: Divide the frequency modulation compensation price series into time periods to obtain multiple trading periods, and extract quantiles and estimate kernel density for the frequency modulation compensation price series in each trading period to obtain the expected return series.

[0027] In this step, the trading period refers to a time unit used for executing independent pricing decisions, defined based on the clearing frequency of the pricing environment. The expected revenue sequence refers to a list of anticipated revenue per unit of electricity estimated within the selected trading time span, combined with the probability distribution of price fluctuations; it can be represented as... .

[0028] Step 201: Based on the preset time duration threshold, divide the frequency modulation compensation price series into multiple trading periods by time dimension.

[0029] In this step, the time duration threshold refers to the pre-set minimum time width used to divide long-term price data into independent analysis units. The value of the time duration threshold is consistent with the trading session division rules of the electricity market, and typical values ​​are 15 minutes, 30 minutes, or 1 hour. Unless otherwise specified, the default value is 1 hour.

[0030] In this embodiment of the application, the frequency modulation compensation price sequence is obtained and represented as a price vector. Based on the preset time duration threshold. Price vector Perform equal-step segmentation. For example, divide a day's price data containing multiple sampling points into several trading periods. The final result is a vector of trading period sets. Each element corresponds to a sub-price vector.

[0031] Step 202: Based on the preset interval step size, classify the frequency adjustment compensation price series in each trading period to obtain multiple price sets.

[0032] In this step, the interval step size refers to the smallest unit distance used to discretize the price value space. The price set refers to the set of sample points whose price values ​​fall within the same interval of step size during a specific trading period, which can be represented as... .

[0033] In this embodiment of the application, the transaction period is extracted. Price data within the range. Based on the preset range step size. The price range within this period is divided into: The process iterates through the price data, assigning sampled points to their corresponding ranges based on their numerical values. Then, sampled points falling within the same range are merged into a price set. The final result is a price set vector. .in Indicates the first A set of sampling points for each interval.

[0034] The interval step size is used to discretize continuous price values, and its value can be determined based on the fluctuation range of historical price data. A simple method is to take (maximum price)... The step size can be 1% to 5% of the minimum price; alternatively, a fixed value can be used, such as 5 yuan / MWh or 10 yuan / MWh. Those skilled in the art can set the step size flexibly according to the actual data distribution.

[0035] Step 203: Use a preset kernel function to fit the probability density of the sampling points falling into each price set to obtain the probability density function of each trading period, and input the price mean of each price set into the probability density function to obtain the price probability distribution value of each trading period.

[0036] In this step, the kernel function refers to a mathematical function used to perform weighted smoothing of local data points during nonparametric estimation, and can include a Gaussian kernel function. The price mean is the arithmetic mean of all sampled points within each price set. The price probability distribution value is the probability of occurrence of the corresponding price point calculated using a probability density function.

[0037] In this embodiment of the application, a preset kernel function is used to process the price set vector. The sampling points are used for probability density fitting. The fitting algorithm uses kernel density estimation. This yields the probability density function for each trading period. Specific bandwidth parameters... The selection of values ​​adopts Silverman's rule of thumb, and is adaptively adjusted in conjunction with the previously extracted price quantile range to ensure the smoothness of the probability distribution description. The specific formula is shown in formula (1): (1) in the formula This represents the probability density function. This indicates the total number of sampling points. This represents the bandwidth parameter. This represents the Gaussian kernel function. Represents the price variable. This represents the price sampling point. Next, the price mean of each price set is calculated and a mean vector is formed. The calculation method for a single element can be expressed as: , in the formula Indicates the first The price mean of a set of prices. Represents a set Price sampling points in the data. This indicates the number of sampling points within the set. The mean vector will then be... The elements in the input are sequentially entered into the probability density function. The calculation is performed to obtain the price probability distribution vector for each trading period. .

[0038] Step 204: Multiply and sum the price probability distribution value and the price mean in the corresponding price set to obtain the expected return for each trading period, and arrange all the expected returns to obtain the expected return sequence.

[0039] In this step, expected return refers to the sum of the products of the price level and its probability of occurrence within a specific trading period.

[0040] In this embodiment of the application, the price mean vector is extracted. With price probability distribution value vector Perform multiplication and addition operations. The calculation formula can be expressed as: , in the formula This indicates the expected return for a single trading session. Indicates the first Price probability distribution values ​​for a set of prices. Indicates the first The average price of each price set. Repeat the above steps in chronological order to calculate the expected return for all trading periods. The final expected return sequence vector is shown below:

[0041] in Indicates the first Expected returns for each trading session.

[0042] Step 103: Perform probability density mapping on the state-of-charge sequence and cumulative cycle lifetime to obtain the margin matrix. Using the preset lifetime loss function, perform loss quantification calculation on the expected revenue sequence and margin matrix to obtain the marginal cost function of the agent corresponding to each distributed energy storage unit.

[0043] In this step, the margin matrix refers to a multi-dimensional data structure that integrates the safety adjustment range of each distributed energy storage unit and the hardware loss status, and can be represented as follows: Each row of this matrix corresponds to an energy storage unit, including its margin value, maximum output, and rated power. The preset lifetime degradation function is a mathematical evaluation formula describing the mapping relationship between the battery's cycle degradation value and its output power and state of charge. An intelligent agent is a computational entity in a distributed architecture that represents an energy storage unit and performs autonomous logical operations and variable updates. The marginal cost function is a differential function model representing the amount of battery depreciation loss caused by a unit power increment.

[0044] Step 301: Based on the preset state division threshold, divide the value range of the state of charge sequence and the cumulative cycle life into multiple numerical intervals, and count the frequency of occurrence of the state of charge sequence and the cumulative cycle life falling into different numerical intervals to obtain multiple operating state values.

[0045] In this step, the state classification threshold refers to a pre-set numerical benchmark used to classify discrete operating state levels. The numerical interval refers to a continuous data segment determined by the state classification threshold. The frequency of occurrence refers to the proportion of sampling points falling within a specific numerical interval. The operating state value refers to a quantitative indicator representing the health status of the distributed energy storage unit, calculated based on the frequency distribution.

[0046] In this embodiment of the application, a sequence of states of charge is obtained. and cumulative cycle life Thresholds are defined based on preset states. The state of charge is divided into multiple numerical ranges. For example, the charge range is divided into high charge ranges. Medium power range and low battery range The frequency of occurrence of statistical sequence data falling into each numerical interval. The calculation formula is as follows: .in This represents the number of samples falling within the interval. The frequency distribution of each interval is statistically analyzed to obtain a state vector composed of multiple operating state values. .

[0047] The state division threshold is used to divide the range of values ​​for state of charge (SOC) and cumulative cycle life into several intervals. For SOC, it is usually divided at equal intervals, for example, every 10% is one interval (0%-10%, 10%-20%, …, 90%-100%). For cumulative cycle life, it can be divided into 5 to 10 intervals based on the battery's rated number of cycles. Unless otherwise specified, the SOC is divided into 10 intervals by default, and the cumulative cycle life is divided into 5 intervals.

[0048] Step 302: Calculate the difference between each operating state value and the preset safety boundary threshold to obtain multiple margin values. Use the margin values ​​to determine the upper and lower limits of power output for each distributed energy storage unit. Arrange all margin values ​​and corresponding upper and lower limits of power output in a matrix according to the numbering order of the distributed energy storage units to obtain a margin matrix.

[0049] In this step, the safety boundary threshold refers to a pre-set limit on the electrical capacity of distributed energy storage units to prevent them from entering deep charge / discharge zones. As a specific implementation, the safety boundary threshold includes a lower limit and an upper limit for the state of charge (SOC). For lithium-ion batteries, the lower limit is typically set at 20%, and the upper limit at 95%. For lead-carbon batteries, the lower limit can be set at 30%, and the upper limit at 90%. Those skilled in the art can determine these limits directly from battery technical manuals or operational safety specifications.

[0050] In this embodiment of the application, each running status value is extracted. Compare it with the preset security boundary threshold. Perform the difference operation. This yields multiple margin values. Specifically, if the running status value is... And the safety boundary threshold is The corresponding margin value is... Next, the upper and lower limits of power output are determined using the margin value and the device hardware parameters.

[0051] For example, based on the margin value Determine the maximum discharge power and maximum charging power Following the numbering order of the distributed energy storage units, the margin values ​​and power boundaries are arranged in a matrix. The final margin matrix is ​​shown below:

[0052] like Figure 2 As shown, Figure 2 This is a flowchart illustrating the method for obtaining the marginal cost function provided in an embodiment of this application.

[0053] Step 311: Using a preset lifetime loss function, numerically map each expected return in the expected return sequence to the corresponding margin value in the margin matrix to obtain the loss change rate of each distributed energy storage unit.

[0054] In this step, the lifetime depreciation function refers to a pre-defined mathematical model used to calculate the physical depreciation of a distributed energy storage unit under unit power output. It can be represented by formula (2): (2) in the formula This represents the rate of change in losses. and These represent the preset lifespan degradation coefficients. Indicates expected return, Indicates the margin value. To provide the real-time output power of the energy storage unit, introduce The term is used to ensure that losses change non-linearly with power, thus preventing the marginal cost obtained from subsequent differentiation from being zero. Numerical mapping refers to the process of transforming variables of different dimensions into a unified dimension through functional correspondences. The rate of change of losses refers to the rate of reduction in battery life caused by fluctuations in unit economic returns.

[0055] It should be noted that the lifetime decay coefficient in formula (2) and and replacement cost It needs to be pre-calibrated based on the actual battery type and operating conditions of the distributed energy storage unit. Among these, The decay coefficient of the squared term of expected return. It is the joint attenuation coefficient of the power square term and the margin value, both of which are dimensionless positive numbers. and The capacity decay rate can be obtained by fitting standard battery cycle aging test data: recording the capacity decay rate at different depths of discharge and number of cycles, and establishing a model based on the expected return. and power output The loss surface with variable is obtained by fitting using the least squares method. , The value of . For lithium-ion batteries, the typical range of values ​​is . , Replacement cost The replacement cost per unit capacity is taken under current market conditions. Those skilled in the art can determine these parameters based on the technical manuals or market quotations of actual energy storage systems.

[0056] In this embodiment of the application, the return value is extracted from the expected return sequence. and the corresponding margin values ​​in the margin matrix Numerical mapping is performed using a pre-defined lifetime loss function. The loss change rate of each unit is calculated. For example, for a distributed energy storage unit... . its expected returns With margin value Input the lifetime loss function. Calculate the loss change rate of the unit. Finally, obtain the loss vector composed of the loss rates of each unit.

[0057] Step 312: Perform value conversion on each loss change rate according to the preset replacement cost to obtain the intermediate loss parameters of each distributed energy storage unit.

[0058] In this step, replacement cost refers to a pre-set value used to measure the investment required to replace a unit of capacity in a distributed energy storage unit. Intermediate loss parameter refers to a cost coefficient that converts the degree of physical wear and tear into economic losses.

[0059] In this embodiment of the application, each rate of change of loss in the loss vector is extracted. For the first... A distributed energy storage unit, with its loss change rate denoted as... According to the preset replacement cost Value conversion is performed to obtain the intermediate loss parameters of this unit. By calculating for each unit in this manner, a vector consisting of the intermediate loss parameters of each unit is finally obtained. .in, The first Intermediate loss parameters of a distributed energy storage unit.

[0060] Step 313: Based on the product of the margin value of the margin matrix and the rated power of the distributed energy storage unit, determine the output power range of each distributed energy storage unit, and construct the first derivative function of each intermediate loss parameter with respect to the output power by means of the numerical correspondence of the intermediate loss parameter within the corresponding output power range.

[0061] In this step, the numerical correspondence refers to the distribution logic of intermediate loss parameters within the power adjustment space. The first derivative function is the marginal function representing the rate of change of total loss with output power.

[0062] In this embodiment of the application, the margin values ​​are extracted from the margin matrix. With the corresponding rated power Calculate their product. This determines the output power range of each unit. Next, intermediate loss parameters are established within this range. With power variables The numerical correspondence is established. An intermediate loss model is constructed and its derivative is calculated. The first derivative function is obtained. This function represents the increment of economic loss for each unit increase in output power at a specific point.

[0063] Step 314: Determine the first derivative function as the marginal cost function of the agent corresponding to each distributed energy storage unit.

[0064] In this embodiment, the calculated first-order derivative function is obtained and used as the marginal cost function for the agent corresponding to each distributed energy storage unit.

[0065] Step 104: Using the marginal cost function as the optimization objective and the margin matrix and the total frequency regulation demand of the power trading system as constraints, the original variables and dual variables are updated and residuals are corrected through alternating direction multiplier optimization until the preset convergence condition is met, so as to obtain the target allocation scheme including the trading volume and the frequency regulation reserved capacity.

[0066] In this step, the optimization objective refers to the mathematical optimization criterion of minimizing the operating losses of each entity while satisfying scheduling safety. Total frequency regulation demand refers to the total amount of regulation power that all participating units are required to provide collaboratively within a specific time period, as mandated by the bidding environment. Constraints refer to the mathematical equations that limit the power output range and business rules such as power balance. Alternating direction multiplier optimization is a distributed iterative algorithm that solves the global problem in parallel by decomposing it and introducing penalty terms.

[0067] The original variables refer to the optimized parameters representing the power output value of the distributed energy storage unit, which can be expressed as: Dual variables are correction coefficients used to reconcile deviations between local decisions and global equilibrium; they can be expressed as... The preset convergence condition refers to the criterion that the global residual norm is less than a set threshold. Traded electricity refers to the power share in the allocation scheme used for energy trading. Frequency regulation reserved capacity refers to the standby power in the allocation scheme reserved for responding to frequency regulation commands. The target allocation scheme refers to the optimal allocation combination of electricity and capacity ultimately determined for each unit.

[0068] like Figure 3 As shown, Figure 3 This is a flowchart illustrating the method for obtaining a target allocation scheme provided in an embodiment of this application.

[0069] Step 401: Set the original variable representing the agent's power output value and the dual variable representing the degree of global imbalance.

[0070] In this step, the original variables refer to the numerical values ​​to be solved, representing the power output level of each distributed energy storage unit in the distributed optimization architecture. The dual variables refer to the coordinating factors used to quantify the global load gap and provide price feedback to the local agents.

[0071] In this embodiment, the original variables are first initialized based on the number of distributed energy storage units participating in the transaction. For example, for five energy storage units in location A, the original variable vector is constructed as follows:

[0072] Simultaneously, dual variables are set according to preset initial values. .

[0073] Step 402: Within the output power range of each distributed energy storage unit, through numerical optimization, search for the desired power value that minimizes the sum of the marginal cost function at the desired power value and the dual variable, and use it as the updated original variable.

[0074] In this step, the power value to be sought refers to every possible power candidate point that the agent attempts to select within the search interval.

[0075] In this embodiment of the application, the output power range determined in the preceding steps is extracted. Within this range, for the power value to be determined Construct an objective function that includes a marginal cost function, dual variables, and a quadratic penalty term. The optimization calculation process is shown in formula (3): (3) in the formula This represents the marginal cost function. This represents the dual variable. This represents the preset iteration step size parameter, which is the penalty parameter that balances the updating of the original variables with global consistency. This represents the power reference value generated in the previous iteration. The goal is to find the power value that minimizes the objective function. This value is then identified as the updated original variable. For example, for a specific energy storage unit... In the interval The internal search finds the power point that minimizes the total loss. Then it will Updated to .

[0076] Penalty parameters Used to balance the updating of original variables with global consistency (i.e., the preset iteration step size parameter), its value directly affects the convergence speed. This application provides two methods for determining it: (1) Empirical value: take A positive number between, or according to Calculation, where It is the sum of the rated power of all distributed energy storage units. The number of agents; (2) Adaptive adjustment: Initially, a small value is taken, and after each iteration, it is dynamically adjusted according to the relationship between the original residual and the dual residual. If the original residual is much larger than the dual residual, it is increased. Conversely, the smaller the value, the lower the value. Those skilled in the art can choose flexibly based on actual computing resources and convergence requirements.

[0077] The above numerical optimization process employs the golden section search algorithm, which is suitable for cases where the objective function is a unimodal function within the interval. The specific optimization steps are as follows: 1. Given output power range ,in For minimum output power, Given the maximum output power; and assuming the convergence accuracy... ,in This refers to the rated power of the distributed energy storage unit. 2. Calculate the golden ratio point , ,in , Candidate power points; 3. If the objective function value ,in The update interval is then Otherwise, update to ; 4. Repeat steps 2-3 until the interval length is less than 1. Take the midpoint of the final interval as the original variable after the update. Those skilled in the art can choose any implementation method based on computing resources and real-time requirements.

[0078] Step 403: Compare the sum of all updated original variables with the total frequency regulation requirement to obtain the residual value.

[0079] In this step, the residual value refers to the algebraic deviation between the total response power of each distributed energy storage unit and the total frequency regulation demand issued by the grid.

[0080] In this embodiment, the original variables of all agents after the current round are obtained. The algebraic sum of all elements in the original variable vector is calculated. Then, the total frequency modulation requirement is read. The sum is then subtracted from the total frequency regulation demand. The calculation formula is shown in formula (4): (4) The residual value This reflects the degree to which the current distributed optimization results balance the global load gap. For example, the sum of the total output of all units is... Total frequency regulation demand is The residual value is .

[0081] Step 404: If the residual value is greater than or equal to the convergence threshold, the dual variable is corrected according to the residual value, and the numerical optimization is performed within the output power range of each distributed energy storage unit until the residual value is less than the convergence threshold to obtain the target original variable.

[0082] In this step, the convergence threshold refers to a pre-set, minimal positive value used to determine whether the algorithm has reached a global equilibrium state. The target original variable refers to the final power allocation command that meets the accuracy requirements after multiple iterations of correction.

[0083] In this embodiment of the application, the calculated residual values ​​are... With the preset convergence threshold Perform a logical comparison. If the residual value is greater than or equal to the convergence threshold... In this case, gradient updates of the dual variable are performed. The correction logic is shown in equation (5): (5) After the correction is complete, the new dual variable will be... Feedback is sent to the optimization step for the next round of calculation. This iterative process continues until the residual value falls within the acceptable error range. The final target original variable vector is shown below:

[0084] Each element represents the agent's optimal output point under the premise of satisfying global balance.

[0085] Step 405: Determine the allocation weight of each agent in the total output power based on the margin value of the margin matrix, calculate the transaction power and frequency regulation reserved capacity based on the target original variables and allocation weights, and construct the target allocation scheme based on the transaction power and frequency regulation reserved capacity.

[0086] In this step, allocation weight refers to the task allocation coefficient determined based on the physical degradation state of each unit. Traded electricity refers to the power portion of the allocation result that participates in the electricity market settlement. Frequency regulation reserve capacity refers to the standby power reserved in the allocation result for responding to frequency fluctuations.

[0087] In the embodiments of this application, the margin values ​​in the margin matrix are used as a basis. Calculate the assigned weight for each agent. The calculation formula is shown in formula (6): (6) in the formula Indicates the first The weights of each agent are calculated. This yields a weight vector composed of the coefficients of each agent. Then, according to the preset allocation ratio coefficient... The target original variables are proportionally segmented to obtain the traded electricity volume and frequency regulation reserved capacity. Specifically, for the target original variables... Its traded electricity volume can be calculated as follows: The reserved capacity for frequency modulation can be calculated as follows: Finally, combine the allocation results of each unit to construct the target allocation scheme.

[0088] The allocation ratio coefficient Used to balance energy trading revenue and frequency regulation reserve revenue, the value range is: This application provides two methods of determination: (1) Fitting based on historical electricity market data: Collect revenue data from the energy market and frequency regulation market over several past trading cycles, and determine the optimal value by using linear regression or grid search with the goal of maximizing total revenue. value; (2) Empirical value: For energy storage systems that primarily provide frequency regulation services, the following value can be taken: For systems primarily reliant on energy arbitrage, the following approach can be adopted: If no prior information is available, the default value will be used. Those skilled in the art can make adjustments based on the actual market environment and energy storage operation objectives.

[0089] Step 105: Based on each target allocation scheme, calculate the marginal rate of change of each agent's revenue relative to the expected revenue sequence through multi-agent reinforcement learning nonlinear mapping, obtain the bid price gradient of each agent, discretize the bid price gradient, and generate a power trading strategy that includes the bid electricity limit, frequency regulation reserved capacity and bid price for multiple trading periods.

[0090] In this step, the marginal revenue change rate refers to the slope of the tangent line to the total revenue function at the point where the currently allocated transaction volume is located. The bid price gradient refers to the unit volume bid slope parameter determined after mapping the revenue change rate to the price range of the bidding environment, which can be expressed as... The upper limit of the declared electricity volume refers to the maximum electricity volume allowed for bidding within the trading period. The declared price refers to the segmented unit price given for each electricity volume interval. The electricity trading strategy refers to the set of bidding messages consisting of volume and price tiers, the upper limit of the electricity volume, and frequency regulation capacity, which can be represented as... .

[0091] Step 501: Construct the revenue function for each agent based on the expected revenue in the expected revenue sequence and the transaction electricity in the target allocation scheme, and calculate the tangent slope of each agent at the corresponding transaction electricity to obtain the revenue change value.

[0092] In this step, the revenue function refers to a mathematical model that represents the mapping relationship between the economic return obtained by a distributed energy storage unit during a specific trading period and the traded electricity volume. The revenue change value refers to a marginal indicator used to reflect the revenue fluctuation caused by changes in unit traded electricity volume.

[0093] In this embodiment of the application, the expected return is extracted from the expected return sequence. and the trading volume in the target allocation scheme First, a revenue function for each agent is constructed based on a pre-defined cost model. The calculation formula can be expressed as: . in the formula This represents the payoff function. This represents the transaction volume variable. This represents the operating cost determined by a pre-defined cost model. The pre-defined cost model refers to a pre-established valuation model used to calculate the hardware depreciation and maintenance costs of the energy storage unit per unit of output. Then, a differential algorithm is used to calculate the revenue function based on the traded electricity volume. The slope of the tangent at that point.

[0094] Among them, the operating cost function A quadratic model is used to reflect the increasing marginal cost characteristic, specifically in the form of: .parameter The nonlinear degradation cost coefficient caused by deep charge and discharge is characterized. It represents the linear component coefficient of fixed operation and maintenance costs allocated to a unit of electricity. It can be obtained by fitting the cycle life curve of the energy storage unit; This can be estimated by dividing the total monthly maintenance cost by the estimated total electricity transaction volume. By default, for rated power... The energy storage unit can be taken , If actual measurement data is lacking, it can be assumed that... If only the linear terms are retained, the payoff function degenerates into a linear form.

[0095] For example, for energy storage units Its trading volume is The change in earnings is obtained by taking the first derivative of the earnings function. The final calculation results of all units are combined. The resulting profit change vector, composed of the profit changes of each unit, is shown below:

[0096] in Indicates the first The change in revenue for each intelligent agent at the corresponding transaction volume.

[0097] Step 502: Map each change in revenue to the corresponding bid price range in the power trading system to obtain multiple bid price gradients.

[0098] In this step, the price range refers to the price boundaries set by the trading environment that allow participants to submit price and volume bids. The price gradient refers to the rate of change of price used to determine the slope of the bid after numerical transformation.

[0099] In this embodiment of the application, the revenue change vector is obtained. Each change in earnings value Next, Mapped to the declared price range The mapping process employs a linear mapping algorithm. Specifically, it utilizes preset mapping coefficients. μ Perform the calculation. The calculation formula can be expressed as: . in the formula This indicates the price gradient for the application. μ The preset mapping coefficient refers to the proportional adjustment parameter used to linearly scale the change in earnings to the declared price range. This represents the mapping offset.

[0100] μ and The method for determining the value is as follows: Based on the lower and upper limits of the bid price allowed by the power trading platform, and the minimum and maximum possible values ​​of the revenue change, a linear correspondence is used to determine the value. That is, the minimum revenue change corresponds to the lower limit of the bid price, and the maximum corresponds to the upper limit of the bid price, with intermediate values ​​mapped proportionally. The range of the revenue change can be obtained through historical data statistics. If no historical data is available, the minimum value can be 0, and the maximum value can be twice the expected maximum revenue. The upper and lower limits of the bid price are directly given by the trading platform's pricing rules, with a typical lower limit of 0 and an upper limit of 1000. Those skilled in the art can determine the value based on the actual platform rules. μ and .

[0101] For example, suppose the power trading platform allows a lower limit of 0 and an upper limit of 1000 for bid prices; the historical minimum value of the change in revenue is 0 and the maximum value is 500. Then take... μ =2, that is, (1000-0) / (500-0)=2. =0. If there is no historical data, the minimum change in profit can be 0, and the maximum can be twice the expected profit (e.g., if the maximum expected profit is 300, then the maximum change in profit is 600). When the declared price range is [0, 1000], μ ≈1.67, =0.

[0102] For example, for a specific change in earnings The declared price gradient is obtained through mapping. The final bid price gradient vector, composed of the bid gradients of each agent, is shown below:

[0103] Step 503: According to the preset power level step size, divide the power output range corresponding to each agent into multiple declared power level intervals, and determine each declared power level interval as a declared power level step.

[0104] In this step, the preset power tier step size refers to a fixed power increment value used to discretize a continuous power range into several declaration levels. The declaration power range refers to the power value range of each segment determined by the power tier step size. The declaration power tier refers to the corresponding discretized power segment in the declaration message.

[0105] In this embodiment of the application, the power output range corresponding to each intelligent agent is obtained. First, based on the preset battery level increments... Divide the range into equally spaced segments. Specifically, calculate the number of segments. Next, each declared electricity volume range is defined as a declared electricity volume tier. Each tier represents an independent declared volume unit.

[0106] For example, for a power range of Energy storage units. According to step size. The resulting step vector of declared electricity levels, composed of various electricity tiers, is shown below:

[0107] in Indicates the first The cumulative power value corresponding to each power level.

[0108] Battery step size The following method is provided for the selection of: (1) Determined according to the ratio of rated power: ,in The desired number of steps is usually taken as... ; (2) Determined by absolute step size: Take The integer (unit consistent with power) is used, and a smaller step size is taken for small-capacity energy storage units; (3) If the power trading platform has a minimum electricity limit for the declared tiers, then that limit value shall be directly used as the step size. By default, for an energy storage unit with a rated power of 100, the step size can be taken as follows: Those skilled in the art can flexibly configure it according to the actual platform requirements.

[0109] Step 511: Based on each declared price gradient, calculate the price value corresponding to each declared electricity range to obtain the declared price ladder for each agent.

[0110] In this step, the declared price tier refers to a price sequence consisting of a series of bid unit prices corresponding to the declared electricity volume tier.

[0111] In this embodiment of the application, the gradient vector of the declared price is obtained. and the declared electricity volume tier vector Based on each declared price tier Calculate the price value corresponding to each declared electricity volume range. The calculation formula can be expressed as: . in the formula Indicates the first The price values ​​for each electricity tier. This represents the preset base bid price, which is the initial reference price used when calculating the bid tiers. The bid price tier for each agent is calculated by iterating through all electricity tiers.

[0112] Basic declared price This refers to the declared price for the first energy tier (when the energy level approaches 0), and its value should reflect the lowest marginal cost of the energy storage unit. This application provides two setting methods: (1) Based on the value of the marginal cost function at zero: ,in This is the derivative of the marginal cost function when the power output is 0. For the lifetime loss function in formula (2), At this point, a positive number needs to be set based on the operation and maintenance costs, which can be taken as... (Linear cost coefficient in step 501); (2) Empirical valuation: Use 1.2 to 1.5 times the local benchmark electricity price for coal-fired power units as the base price for the bid. Unless otherwise specified, this can be the default. At this point, the bid price is entirely determined by the price tier. Those skilled in the art can adjust it flexibly according to market strategies.

[0113] For example, for intelligent agents Combined with its declared price tiers and the power step vector The corresponding price step vector is shown below:

[0114] Step 512: Take the transaction volume in each target allocation scheme as the upper limit of the declared volume, and combine the upper limit of the declared volume, the declared volume tier, the declared price tier, and the frequency regulation reserved capacity corresponding to each agent based on the time sequence of the transaction period to obtain the power trading strategy corresponding to each agent.

[0115] In this step, the upper limit for the declared electricity volume refers to the maximum amount of electricity allowed to be bid on within a single trading session.

[0116] In this embodiment, the transaction volume in each target allocation scheme is obtained and confirmed as the upper limit of the declared volume. Next, based on the time sequence of the trading period, the data for each agent's declared electricity limit, declared electricity tier, declared price tier, and frequency regulation reserved capacity are combined. The combination process is executed according to a preset communication protocol format, which refers to a pre-defined message structure used to encapsulate the declared data into a message structure that conforms to the trading platform's receiving standards.

[0117] For example, for trading sessions Set the battery limit. FM reserved capacity , Application for electricity volume tiered vector and the tiered vector of declared prices Logical associations are performed. The final electricity trading strategy matrix for each agent is shown below:

[0118] The strategy matrix is ​​sent to the power trading platform via a network interface, thereby completing the quantity and price declaration for that period.

[0119] As a feasible example, the preset communication protocol format adopts a JSON message structure, which is compatible with the standard data interface specification of the power trading platform. The power trading strategy message generated by each agent includes the following fields: unique identifier of the energy storage unit, time interval of the trading period, upper limit of the declared electricity volume, frequency regulation reserved capacity, and a tiered array of declared electricity volumes. Each element in the tiered array of declared electricity volumes contains two subfields: the electricity volume tier value and the corresponding declared price. Those skilled in the art can adjust the field naming and numerical format according to the requirements of the actual trading platform.

[0120] This application's embodiments ensure the comprehensiveness of decision-making by collecting internal physical state data of energy storage units and external market price signals. This solves the problem of poor strategy stability caused by insufficient representation of price signals in existing solutions. It achieves accurate conversion of the internal physical health state of energy storage equipment into the economic cost domain, avoiding nonlinear accelerated degradation of lifespan due to frequent frequency regulation. It achieves efficient convergence of resource allocation among multiple agents, achieving a balance between overall system economy and individual physical constraints while meeting the overall frequency regulation requirements of the power trading system. This enhances the executability of the application strategy.

[0121] Figure 4 This is a schematic diagram of a specific implementation of the multi-agent-based power trading strategy generation system provided in this application, with reference to... Figure 4 The system may include: The acquisition module 21 is used to acquire the state of charge sequence and cumulative cycle life of each distributed energy storage unit, as well as the frequency regulation compensation price sequence of the power trading system. The segmentation module 22 is used to divide the frequency modulation compensation price series into time periods to obtain multiple trading periods, and to extract quantiles and estimate kernel density for the frequency modulation compensation price series in each trading period to obtain the expected return series. The mapping module 23 is used to perform probability density mapping on the state of charge sequence and cumulative cycle lifetime to obtain the margin matrix. Using the preset lifetime loss function, the module performs loss quantification calculation on the expected revenue sequence and margin matrix to obtain the marginal cost function of the agent corresponding to each distributed energy storage unit. The update module 24 is used to update the original variables and dual variables and correct the residuals by using the marginal cost function as the optimization objective and the margin matrix and the total frequency regulation demand of the power trading system as constraints, through alternating direction multiplier optimization, until the preset convergence condition is met, so as to obtain the target allocation scheme including the trading volume and the frequency regulation reserved capacity. The calculation module 25 is used to calculate the marginal rate of change of each agent's revenue relative to the expected revenue sequence based on each target allocation scheme through multi-agent reinforcement learning nonlinear mapping, obtain the bid price gradient of each agent, discretize the bid price gradient, and generate a power trading strategy that includes the bid electricity limit, frequency regulation reserved capacity and bid price for multiple trading periods.

[0122] The multi-agent-based power trading strategy generation system of this application is used to implement the aforementioned multi-agent-based power trading strategy generation method. Therefore, the specific implementation of the multi-agent-based power trading strategy generation system can be found in the embodiment section of the multi-agent-based power trading strategy generation method above. The specific implementation can be referred to the description of the corresponding embodiment, which will not be repeated here.

[0123] Figure 5 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0124] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the multi-agent-based power trading strategy generation method described above.

[0125] The electronic device may include a processor 510 and a memory 520 storing computer program instructions.

[0126] Specifically, the processor 510 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0127] Memory 520 may include mass storage for data or instructions. For example, and not limitingly, memory 520 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 520 may include removable or non-removable (or fixed) media. Where appropriate, memory 520 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 520 is non-volatile solid-state memory.

[0128] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to the first aspect of this disclosure.

[0129] The processor 510 reads and executes computer program instructions stored in the memory 520 to implement any of the multi-agent-based power trading strategy generation methods in the above embodiments.

[0130] In one example, the electronic device may also include a communication interface 530 and a bus 540. Wherein, such as Figure 5 As shown, the processor 510, memory 520, and communication interface 530 are connected through bus 540 and complete communication with each other.

[0131] The communication interface 530 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0132] Bus 540 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 540 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0133] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described multi-agent-based power trading strategy generation methods.

[0134] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0135] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the embodiments of the multi-agent-based power trading strategy generation method described above.

[0136] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0137] The multi-agent-based power trading strategy generation method and system provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for generating power trading strategies based on multi-agent systems, characterized in that, include: Obtain the state-of-charge sequence and cumulative cycle lifetime of each distributed energy storage unit, as well as the frequency regulation compensation price sequence of the power trading system; The frequency modulation compensation price series is divided into time periods to obtain multiple trading periods. The frequency modulation compensation price series in each trading period is then subjected to quantile extraction and kernel density estimation to obtain the expected return series. The state-of-charge sequence and the cumulative cycle lifetime are subjected to probability density mapping to obtain a margin matrix. Using a preset lifetime loss function, the expected revenue sequence and the margin matrix are subjected to loss quantification calculation to obtain the marginal cost function of the agent corresponding to each distributed energy storage unit. Using the marginal cost function as the optimization objective and the margin matrix and the total frequency regulation demand of the power trading system as constraints, the original variables and dual variables are updated and residuals are corrected through alternating direction multiplier optimization until the preset convergence condition is met, resulting in a target allocation scheme that includes the traded electricity volume and the frequency regulation reserved capacity. Based on each target allocation scheme, the marginal rate of change of each agent's revenue relative to the expected revenue sequence is calculated through multi-agent reinforcement learning nonlinear mapping to obtain the bid price gradient of each agent. The bid price gradient is then discretized to generate a power trading strategy that includes the bid electricity limit, frequency regulation reserved capacity, and bid price for multiple trading periods.

2. The method for generating power trading strategies based on multi-agent systems according to claim 1, characterized in that, The frequency modulation compensated price series is divided into time periods to obtain multiple trading periods. For each trading period, quantile extraction and kernel density estimation are performed on the frequency modulation compensated price series to obtain the expected return series, including: Based on a preset time duration threshold, the frequency modulation compensation price sequence is divided into multiple trading periods by time dimension. Based on a preset interval step size, the frequency adjustment compensation price series within each trading period is categorized to obtain multiple price sets; The probability density function of each trading period is obtained by fitting the sampling points falling into each price set using a preset kernel function, and the price mean of each price set is input into the probability density function to obtain the price probability distribution value of each trading period. The expected return for each trading period is obtained by multiplying and summing each price probability distribution value with the mean price in the corresponding price set, and then arranging all the expected returns to obtain the expected return sequence.

3. The method for generating power trading strategies based on multi-agent systems according to claim 1, characterized in that, A probability density mapping is performed on the charged state sequence and the cumulative cycle lifetime to obtain a margin matrix, including: Based on a preset state division threshold, the range of values ​​for the state of charge sequence and the cumulative cycle life is divided into multiple numerical intervals, and the frequency of occurrence of the state of charge sequence and the cumulative cycle life falling into different numerical intervals is statistically analyzed to obtain multiple operating state values. The difference between each operating state value and the preset safety boundary threshold is calculated to obtain multiple margin values. The upper and lower limits of power output for each distributed energy storage unit are determined using the margin values. According to the numbering order of the distributed energy storage units, all margin values ​​and their corresponding upper and lower limits of power output are arranged in a matrix to obtain a margin matrix.

4. The method for generating power trading strategies based on multi-agent systems according to claim 1, characterized in that, Using a preset lifetime loss function, loss quantification calculations are performed on the expected revenue sequence and the margin matrix to obtain the marginal cost function of the agent corresponding to each distributed energy storage unit, including: By using a preset lifetime loss function to numerically map each expected return in the expected return sequence to the corresponding margin value in the margin matrix, the loss change rate of each distributed energy storage unit is obtained. The value of each loss change rate is converted based on the preset replacement cost to obtain the intermediate loss parameters of each distributed energy storage unit. The output power range of each distributed energy storage unit is determined by multiplying the margin value of the margin matrix with the rated power of the distributed energy storage unit. The first derivative function of each intermediate loss parameter with respect to the output power is constructed by the numerical correspondence of the intermediate loss parameter within the corresponding output power range. The first derivative function is determined as the marginal cost function of the agent corresponding to each distributed energy storage unit.

5. The method for generating power trading strategies based on multi-agents according to claim 4, characterized in that, Using the marginal cost function as the optimization objective and the margin matrix and the total frequency regulation demand of the power trading system as constraints, the original and dual variables are updated and residuals are corrected through alternating direction multiplier optimization until the preset convergence condition is met, resulting in a target allocation scheme including traded electricity volume and frequency regulation reserved capacity, including: Set the primary variable to represent the agent's power output value and the dual variable to represent the degree of global imbalance; Within the output power range of each distributed energy storage unit, numerical optimization is performed to search for the desired power value that minimizes the sum of the marginal cost function at the desired power value and the dual variable, which is then used as the updated original variable. The sum of all updated original variables is compared with the total frequency regulation requirement to obtain the residual value; If the residual value is greater than or equal to the convergence threshold, the dual variable is corrected according to the residual value, and numerical optimization is performed within the output power range of each distributed energy storage unit until the residual value is less than the convergence threshold to obtain the target original variable. The allocation weight of each agent in the total output power is determined based on the margin value of the margin matrix. The transaction power and frequency regulation reserved capacity are calculated based on the target original variables and the allocation weight. The target allocation scheme is constructed based on the transaction power and the frequency regulation reserved capacity.

6. The method for generating power trading strategies based on multi-agents according to claim 5, characterized in that, Based on each target allocation scheme, the bid price gradient for each agent is obtained by calculating the marginal rate of change of each agent's revenue relative to the expected revenue sequence, including: Based on the expected returns in the expected return sequence and the transaction volume in the target allocation scheme, construct the return function corresponding to each agent, and calculate the tangent slope of each agent at the corresponding transaction volume to obtain the return change value. Each change in revenue is mapped to the corresponding bid price range in the power trading system to obtain multiple bid price gradients; According to the preset power level step size, the power output range corresponding to each intelligent agent is divided into multiple declared power level intervals, and each declared power level interval is determined as a declared power level step.

7. The method for generating power trading strategies based on multi-agent systems according to claim 6, characterized in that, Discretize the declared price gradient to generate a power trading strategy that includes the upper limit of declared electricity volume, frequency regulation reserved capacity, and declared price for multiple trading periods, including: Based on each declared price gradient, calculate the price value corresponding to each declared electricity range to obtain the declared price tier for each agent; The transaction volume in each target allocation scheme is used as the upper limit of the declared volume. Based on the time sequence of the transaction period, the upper limit of the declared volume, the declared volume tier, the declared price tier, and the frequency regulation reserved capacity corresponding to each agent are combined to obtain the power trading strategy corresponding to each agent.

8. A power trading strategy generation system based on multi-agent systems, characterized in that, include: The acquisition module is used to acquire the state of charge sequence and cumulative cycle life of each distributed energy storage unit, as well as the frequency regulation compensation price sequence of the power trading system; The segmentation module is used to divide the frequency modulation compensation price series into time periods to obtain multiple trading periods, and to extract quantiles and estimate kernel density for the frequency modulation compensation price series in each trading period to obtain the expected return series. The mapping module is used to perform probability density mapping on the state of charge sequence and the cumulative cycle lifetime to obtain a margin matrix. Using a preset lifetime loss function, the module performs loss quantification calculation on the expected revenue sequence and the margin matrix to obtain the marginal cost function of the agent corresponding to each distributed energy storage unit. The update module is used to update the original variables and dual variables and correct the residuals by means of alternating direction multiplier optimization, with the marginal cost function as the optimization objective and the margin matrix and the total frequency regulation demand of the power trading system as constraints, until the preset convergence condition is met, so as to obtain the target allocation scheme including the trading volume and the frequency regulation reserved capacity. The calculation module is used to calculate the marginal rate of change of each agent's revenue relative to the expected revenue sequence based on each target allocation scheme through multi-agent reinforcement learning nonlinear mapping, obtain the bid price gradient of each agent, discretize the bid price gradient, and generate a power trading strategy that includes the bid electricity limit, frequency regulation reserved capacity and bid price for multiple trading periods.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the multi-agent-based power trading strategy generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the generation method of a multi-agent power trading strategy as described in any one of claims 1 to 7.

Images