Power parameter auxiliary model decision method and device based on multi-agent system

Abstract

This application relates to a power parameter auxiliary model decision-making method and apparatus based on a multi-agent system. The method includes: acquiring historical sample data for several days, summarizing it into a sample dataset, and preprocessing it; analyzing the sample dataset to obtain the adjustment ratio range; acquiring the original feature dataset through hot encoding and expanding it to obtain an expanded feature dataset; training a multi-agent deep reinforcement learning model using the sample dataset, adjustment ratio range, and expanded feature dataset as input data; training the multi-agent deep reinforcement learning model multiple times and selecting the best model; and calculating the optimal declaration parameters based on the best model. Multi-source data fusion provides comprehensive and accurate information support. Combined with the multi-agent deep reinforcement learning model, it seeks the intrinsic relationship between different declaration strategies and power generation benefits, optimizing multi-dimensional and multi-period power trading decisions, improving decision optimization efficiency and the accuracy of the final decision.

Description

Technical Field

[0001] This application relates to the field of multi-agent system application technology in electricity market transactions, specifically to a decision-making method and device for power parameter auxiliary models based on multi-agent systems. Background Technology

[0002] Currently, according to the rules of the electricity spot market, new energy power generation companies participating in electricity spot trading are generally required to submit day-ahead declarations as required by market rules. This means that market participants submit their power generation and consumption plans, bids, and other information for the operating day to the electricity trading institution or dispatching agency one day in advance (generally the day before the operating day). This provides market participants with an opportunity to plan ahead and participate in market competition, helping to improve the efficiency of electricity market declarations and the rationality of resource allocation.

[0003] Currently, most new energy power generation companies use three main methods for their day-ahead reporting: the first is to directly use the raw power forecast data from the wind power forecasting system as the market reporting plan; the second is to manually adjust the raw power forecast data to form the market reporting plan; and the third is to use software systems to generate the market reporting plan. However, the first method relies entirely on the raw power forecast data, failing to comprehensively consider various factors such as medium- and long-term contract conditions, market supply and demand, and market assessment rules. Furthermore, due to significant deviations in the power forecast results, the reporting plan lacks comprehensive consideration of factors and in-depth data analysis and mining, resulting in an insufficiently comprehensive market reporting plan. The second method, although it makes some adjustments to the raw power forecast data, relies excessively on individual abilities and experience, without a clear quantitative processing of the intermediate steps. In the actual market reporting plan, some factors are subject to human randomness, and the process is inefficient. The third approach, while utilizing information technology, is based on historical transaction data. It employs statistical analysis to analyze price trends and performs error analysis between short-term power forecasts and actual power generation. The error results are used as the basis for short-term power adjustments to generate market declaration plans. This approach only uses conventional mathematical statistics methods, has simple logic, does not fully consider data correlation, and lacks in-depth analysis and mining of the full dataset. This results in low accuracy of the generated market declaration plans, making it difficult to maximize the reduction of power generation costs and improve power generation efficiency. It is also detrimental to improving the efficiency and accuracy of power market declarations. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, this application provides a power parameter auxiliary model decision-making method and apparatus based on a multi-agent system, specifically adopting the following technical solution:

[0005] A power parameter-assisted model decision-making method based on a multi-agent system includes the following steps:

[0006] Step 1: Obtain sample data of the electricity market corresponding to the collection time t on day d, summarize them into a sample dataset, and preprocess the sample dataset; the sample dataset includes at least the day-ahead price, real-time price, market day-ahead clearing price, market real-time clearing price, clearing power data, short-term power data, adjustment ratio data, and environmental parameters; where d = 1, 2, ..., D, and D is the total number of days in the sample data;

[0007] Step 2: Analyze the existing adjustment ratio data in the sample dataset to obtain the adjustment ratio range;

[0008] Step 3: Calculate the difference between the day-ahead price and the real-time price in the electricity market based on the sample dataset, and process the difference between the day-ahead price and the real-time price into the original feature dataset through hot coding. Then, expand the original feature dataset by window sliding processing to obtain the expanded feature dataset.

[0009] Step 4: Using the sample dataset, the adjusted scale range, and the expanded feature dataset as input data, train a multi-agent deep reinforcement learning model with settlement revenue and profit recovery loss as reward functions;

[0010] Step 5: Perform multiple training sessions based on the multi-agent deep reinforcement learning model, select the best multi-agent deep reinforcement learning model, obtain the optimal adjustment ratio based on the best multi-agent deep reinforcement learning model, and obtain the optimal application parameters based on the optimal adjustment ratio and short-term power.

[0011] Optionally: When obtaining sample data of the electricity market in step one, the data is collected in days and each day is divided into several equally spaced collection times, and sample data is obtained for each collection time.

[0012] Optional: In step one, a data normalization method is used to preprocess the sample dataset.

[0013] First, obtain the maximum and minimum values ​​of different sample data in the sample dataset, and then classify the sample data according to different times of the same day to obtain the sample units for day d.

[0014] The difference between the sample data at time t is obtained based on the maximum and minimum values ​​of the corresponding sample data.

[0015] Based on the sample unit of day d, obtain the sample data at time t in day d, and obtain the normalized sample data corresponding to time t in day d based on the difference between the sample data at time t in day d and the sample data corresponding to time t in day d.

[0016] Optionally: Step two, which involves analyzing existing adjustment ratio data in the sample dataset to obtain the adjustment ratio range, includes:

[0017] Convert the cleared power data corresponding to the collection time t on day d in the sample dataset into power system output data;

[0018] Calculate the adjustment ratio between the power system output data and the short-term power data at the time of data collection on day d at time t.

[0019] The maximum and minimum values ​​of the adjustment ratio are obtained by statistical analysis of the sample dataset using statistical methods, and the maximum and minimum values ​​of the adjustment ratio are used as the range of the adjustment ratio.

[0020] The probability distribution of the adjustment ratio at each collection time in the historical days is calculated using a probability density curve function.

[0021] Optionally: Step three, which involves processing the difference between the day-to-day price and the real-time price into feature data using a hot coding method, includes:

[0022] Obtain the day-ahead price and real-time price corresponding to the data collection time t on day d;

[0023] Calculate the difference between the day-ahead price and the real-time price at time t on day d;

[0024] The difference between the previous day's price and the real-time price at time t on day d is compared with the threshold 0 to determine the direction of the price difference: when the difference between the previous day's price and the real-time price is less than 0, the direction of the price difference is -1; when the difference between the previous day's price and the real-time price is equal to 0, the direction of the price difference is 0; when the difference between the previous day's price and the real-time price is greater than 0, the direction of the price difference is 1.

[0025] The obtained price difference directions are summarized to obtain characteristic data.

[0026] Optionally, step three, which involves expanding the original feature dataset using a window sliding process to obtain an expanded feature dataset, includes:

[0027] Set the window width W and sliding step size ΔW for the time window, and arrange the feature data of the original feature dataset in chronological order to obtain the sorted original feature dataset;

[0028] The starting position of the sorted original feature dataset is shifted one sliding step ΔW to the right according to the window width W of the time window, and the feature data at the time window position is set to 0 to obtain new feature data.

[0029] Optionally: The steps in step four of establishing the multi-agent deep reinforcement learning model, which use settlement of income and recovery of losses as reward functions, include:

[0030] Obtain the actual power generation, cleared power data, and real-time market clearing price corresponding to the data collection time t on day d;

[0031] The deviation range of the cleared power data at the corresponding time is calculated based on the cleared power data;

[0032] Determine whether the actual power generation at time t on day d is within the deviation range of the cleared power data: if the actual power generation is within the deviation range of the cleared power data, then set the corresponding deviation recovery loss to 0; if the actual power generation is not within the deviation range of the cleared power data, then calculate the deviation recovery loss at time t.

[0033] The deviation recovery loss on day d is calculated based on the deviation recovery loss corresponding to different acquisition times.

[0034] The settlement revenue at time t on day d is calculated based on the actual power generation, cleared power data, day-ahead clearing price, and real-time clearing price of the market at time t during day d.

[0035] The settlement revenue for day d is calculated based on the settlement revenue collected at time t on day d.

[0036] The reward function value for day d is obtained based on the difference between the deviation recovery loss and the settlement revenue on day d.

[0037] Optionally: The method for calculating the deviation recovery loss includes:

[0038]

[0039] J t Let be the deviation recovery loss at acquisition time t; T be the total number of acquisition times; λ be the market-allowed deviation ratio coefficient; Q_zhong t P_zhong represents the total electricity volume of the medium- and long-term contracts at time t. t P_riqian represents the medium- to long-term contract weighted electricity price at time t. t Pzr is the market-day clearing price at time t. t Q_shishi represents the medium- to long-term and day-ahead weighted electricity price at time t. t P_shishi represents the actual power generation at time t. t Q_riqian represents the real-time market clearing price at time t. t Q_short represents the cleared power data reported at time t.t α represents the short-term power data at acquisition time t; t This refers to the adjustment ratio data reported at time t.

[0040] Optionally: The method for calculating the settlement revenue includes:

[0041]

[0042] J jiesuan To summarize the revenue; T is the total number of data collection times in the input model; J_jiesuan t The settlement revenue at time t; Q_zhong t P_zhong represents the total electricity volume of the medium- and long-term contracts at time t. t P_riqian represents the medium- to long-term contract weighted electricity price at time t. t The market-day clearing price at time t; Q_shishi t P_shishi represents the actual power generation at time t. t Q_riqian represents the real-time market clearing price at time t. t Q_short represents the cleared power data reported at time t. t α represents the short-term power data at acquisition time t; t This refers to the adjustment ratio data reported at time t.

[0043] Furthermore, this application also discloses a power parameter auxiliary model decision-making device based on a multi-agent system, the device comprising:

[0044] The sample data acquisition module is used to acquire sample data of the electricity market corresponding to the collection time t on day d and summarize them into a sample dataset, and preprocess the sample dataset; the sample dataset includes at least day-ahead price, real-time price, market day-ahead clearing price, market real-time clearing price, clearing power data, short-term power data, adjustment ratio data and environmental parameters; where d = 1, 2, ..., D, and D is the total number of days in the sample data;

[0045] The adjustment range determination module is used to analyze the existing adjustment ratio data in the sample dataset to obtain the adjustment ratio range;

[0046] The feature data augmentation module is used to calculate the difference between the day-ahead price and the real-time price in the electricity market based on the sample dataset, and to process the difference between the day-ahead price and the real-time price into the original feature dataset through hot encoding. Then, the original feature dataset is augmented by window sliding processing to obtain the augmented feature dataset.

[0047] The model building module is used to build a multi-agent deep reinforcement learning model with settlement income and profit recovery loss as reward functions, using sample datasets, adjusted scale ranges and expanded feature datasets as input data.

[0048] The application strategy determination module is used to perform multiple trainings based on a multi-agent deep reinforcement learning model, select the best multi-agent deep reinforcement learning model, obtain the optimal adjustment ratio based on the best multi-agent deep reinforcement learning model, and obtain the optimal application parameters based on the optimal adjustment ratio and short-term power.

[0049] Beneficial effects

[0050] The technical solution of this application achieves the following beneficial effects:

[0051] The power parameter-assisted model decision-making method of this application provides comprehensive and accurate information support through multi-source data fusion technology. Combined with a multi-agent deep reinforcement learning model, it seeks the intrinsic relationship between different declaration strategies and power generation benefits. It can optimize power trading decisions in multiple time periods, flexibly respond to complex and ever-changing market environments, quantify day-ahead declaration issues, eliminate the influence of human factors, and further explore the correlation of data. The constructed auxiliary decision-making model can determine the optimal operating strategy to improve the declaration efficiency and accuracy of the power market. Attached Figure Description

[0052] Figure 1 This is a flowchart of the power parameter-assisted model decision-making method in the embodiments of this application.

[0053] Figure 2 This is a structural diagram of the power parameter auxiliary model decision-making device in the embodiments of this application.

[0054] Figure 3 This is a schematic diagram illustrating the changes in the adjustment ratio, short-term power, and optimal application parameters in the embodiments of this application.

[0055] Figure 4 This is a structural diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0056] The present application will now be further described with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application and should not be construed as limiting the scope of protection of the present application. It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present application.

[0057] Combination Figure 1 As shown in the embodiment, this application specifically discloses a power parameter auxiliary model decision-making method based on a multi-agent system, which includes the following steps:

[0058] Step 1: Obtain sample data of the electricity market corresponding to the collection time t on day d, summarize them into a sample dataset, and preprocess the sample dataset; the sample dataset includes at least the day-ahead price, real-time price, market day-ahead clearing price, market real-time clearing price, clearing power data, short-term power data, adjustment ratio data, and environmental parameters; where d = 1, 2, ..., D, and D is the total number of days in the sample data;

[0059] Step 2: Analyze the existing adjustment ratio data in the sample dataset to obtain the adjustment ratio range;

[0060] Step 3: Calculate the difference between the day-ahead price and the real-time price in the electricity market based on the sample dataset, and process the difference between the day-ahead price and the real-time price into the original feature dataset through hot coding. Then, expand the original feature dataset by window sliding processing to obtain the expanded feature dataset.

[0061] Step 4: Using the sample dataset, the adjusted scale range, and the expanded feature dataset as input data, train a multi-agent deep reinforcement learning model with settlement revenue and profit recovery loss as reward functions;

[0062] Step 5: Perform multiple training sessions based on the multi-agent deep reinforcement learning model, select the best multi-agent deep reinforcement learning model, obtain the optimal adjustment ratio based on the best multi-agent deep reinforcement learning model, and obtain the optimal application parameters based on the optimal adjustment ratio and short-term power.

[0063] Specifically, in step one of this application's embodiments, when acquiring sample data from the electricity market, the data is collected on a daily basis, with each day divided into several equally spaced collection periods. Sample data is acquired for each collection period. Preferably, this embodiment uses a daily basis with 15-minute intervals for data collection and processing, resulting in 96 collection periods per day.

[0064] Furthermore, since the collected raw sample data are based on different data units and contain some dirty data (such as inaccurate, incomplete, inconsistent, or non-standard data), data cleaning is necessary. In step one of this embodiment, a data normalization method is used to preprocess the sample dataset to avoid bias caused by differences in data levels affecting the model's generalization ability. By standardizing the data to a fixed range, such as [0,1] or [-1,1], the model training speed is accelerated and the prediction accuracy is improved. The specific preprocessing steps are as follows:

[0065] First, obtain the maximum and minimum values ​​of different sample data in the sample dataset, and then classify the sample data according to different times of the same day to obtain the sample unit for day d. It should be noted that the maximum and minimum values ​​obtained are selected from the range of sample data of the same type within the sample dataset. For example, when obtaining the maximum and minimum values ​​of a certain sample data, you can obtain the sample data of the same type collected in the historical days, and then filter out the maximum and minimum values ​​based on the sample data in the historical days.

[0066] The difference between the sample data at time t is obtained based on the maximum and minimum values ​​of the corresponding sample data.

[0067] Based on the sample units of day d, obtain the sample data at time t in day d, and obtain the normalized sample data corresponding to time t in day d based on the difference between the sample data at time t in day d and the sample data corresponding to time t in day d:

[0068]

[0069] Where X' d,t X represents the normalized sample data collected at time t on day d; d,t This refers to the sample data collected at time t on day d. The minimum value of the sample data at time t in the sample dataset; It represents the maximum value of the sample data at time t in the sample dataset.

[0070] More specifically, in step two of this application embodiment, the adjustment ratio range is obtained by analyzing the existing adjustment ratio data in the sample dataset. The adjustment ratio is a crucial decision-making factor in the electricity market, requiring market participants to comprehensively consider factors such as market price fluctuations, changes in supply and demand, their own costs and benefits, and policy and regulatory changes, in order to maximize economic benefits and ensure stable operation of the power system. In this embodiment, the allowable range of the adjustment ratio is determined by analyzing the collected historical sample data, and adjustments are then made within this range. The specific steps include:

[0071] First, the cleared power data corresponding to the collection time t on day d in the sample dataset is converted into power system output data;

[0072] Then, the adjustment ratios for the power system output data and short-term power data corresponding to the data collection time t on day d are calculated respectively:

[0073]

[0074] Where R d,t The adjustment ratio for the data collection time t on day d; Power_cleand,t This refers to the power system output data collected at time t on day d; Power_short d,t This refers to the short-term power data collected at time t on day d.

[0075] Then, statistical methods were used to analyze the sample dataset to obtain the maximum value of the adjustment ratio, R. max and minimum value R min The maximum and minimum values ​​of the adjustment ratio are used as the adjustment ratio range R. range =[R min ,R max ];

[0076] Finally, the probability distribution of the adjustment ratio for each collection time in the historical days is calculated using the probability density curve function to obtain the optimal adjustment ratio for each collection time.

[0077] In detail, in step three of this application's embodiments, the difference between the day-to-day price and the real-time price is processed into feature data using a hot coding method. Hot coding is an encoding method that converts categorical variables into numerical variables. It represents each category as a binary vector, avoiding the problem of comparing numerical values ​​between different categories and enabling the model to better handle categorical variables. Its specific steps include:

[0078] First, obtain the day-ahead price and real-time price corresponding to the data collection time t on day d;

[0079] Then, the difference between the day-ahead price and the real-time price at time t on day d is calculated, i.e.:

[0080] Price_diff d,t =Price_riqian d,t -Price_shishi d,t ;

[0081] Then, the difference between the previous day's price and the real-time price at time t on day d is compared with a threshold of 0 to determine the direction of the price difference: when the difference between the previous day's price and the real-time price is less than 0, the direction of the price difference is -1; when the difference between the previous day's price and the real-time price is equal to 0, the direction of the price difference is 0; when the difference between the previous day's price and the real-time price is greater than 0, the direction of the price difference is 1. The process of determining the direction of the price difference is shown in the following formula:

[0082]

[0083] Among them, Price_diff d,t Price_riqian represents the price difference at time t on day d. d,tPrice_shishi is the day-ahead price at time t on day d. d,t Price_hot represents the real-time price at time t on day d. d,t The direction of the price difference at time t on day d;

[0084] Finally, by summing up the obtained price difference directions, the corresponding feature data can be obtained.

[0085] Furthermore, in step three of this application embodiment, the original feature dataset is subjected to window sliding processing to expand it into an expanded feature dataset. The specific steps include:

[0086] First, set the window width W and sliding length ΔW of the time window, and then arrange the feature data of the original feature dataset in chronological order to obtain the sorted original feature dataset.

[0087] The starting position of the sorted original feature dataset is shifted one sliding step ΔW to the right according to the window width W of the time window, and the feature data at the time window position is set to 0 to obtain new feature data.

[0088] In this embodiment, a time window of 96 points is used to perform lag processing on the existing feature dataset. Each feature data point is shifted backward by one time window length to become the new feature data, based on the time window length. For example, as shown in the figure, feature1 = [1,2,3,4,5,6,...]. Assuming the time window length W is 3, feature1 is shifted backward by W points sequentially. That is, data point 1 is moved to the position of data point 4, data point 2 is moved to the position of data point 5, data point 3 is moved to the position of data point 6, and so on, resulting in feature2 = [0,0,0,1,2,3,...]. The first T data points of feature2 are filled with 0.

[0089] More specifically, the step in step four of this application's embodiments, which uses settlement revenue and profit recovery loss as reward functions, includes the following steps when establishing a multi-agent deep reinforcement learning model:

[0090] Obtain the actual power generation, cleared power data, and real-time market clearing price corresponding to the data collection time t on day d;

[0091] The deviation range of the cleared power data at the corresponding time is calculated based on the cleared power data;

[0092] Determine whether the actual power generation at time t on day d is within the deviation range of the cleared power data: if the actual power generation is within the deviation range of the cleared power data, then set the corresponding deviation recovery loss to 0; if the actual power generation is not within the deviation range of the cleared power data, then calculate the deviation recovery loss at time t.

[0093] For example, when Q_riqian t >(1+λ)×Q_shishi t When the actual power generation Q_shishit is lower than the cleared power generation data Q_riqian t The lower limit of the allowable deviation, at which point the deviation recovery loss is:

[0094] J 11，t =max(Q_riqian t -(1+λ)×Q_shishi t ,0)×max(Pzr t -P_shishi t ,0);

[0095] When Q_riqian t <(1-λ)×Q_shishi t At that time, the actual power generation Q_shishit is higher than the cleared power generation data Q_riqian. t The upper limit of the allowable deviation, at which point the deviation recovery loss is:

[0096] J 12，t =max(Q_riqian t -(1+λ)×Q_shishi t ,0)×max(Pzr t -P_shishi t ,0);

[0097] The deviation recovery loss on day d is calculated based on the deviation recovery loss corresponding to different acquisition times. The calculation method for the deviation recovery loss includes:

[0098]

[0099] J t In this embodiment, J represents the deviation recovery loss at acquisition time t. t =J 11,t +J 12,t T represents the total number of data collection points; λ represents the market-acceptable deviation ratio coefficient; Q_zhong t P_zhong represents the total electricity volume of the medium- and long-term contracts at time t. tP_riqian represents the medium- to long-term contract weighted electricity price at time t. t The market-day clearing price at time t; Q_riqian t Pzr represents the cleared energy data at time t. t Q_shishi represents the medium- to long-term and day-ahead weighted electricity price at time t. t P_shishi represents the actual power generation at time t. t The real-time market clearing price at time t; Q_riqian t Q_short represents the cleared power data reported at time t. t α represents the short-term power data at acquisition time t; t This refers to the adjustment ratio data reported at time t.

[0100] Calculate the settlement revenue at time t on day d based on the actual power generation, cleared power data, day-ahead market clearing price, and real-time market clearing price:

[0101]

[0102] The settlement revenue for day d is calculated based on the settlement revenue collected at time t on day d:

[0103]

[0104] J jiesuan To summarize the revenue; T is the total number of data collection times in the input model; J_jiesuan t The settlement revenue at time t; Q_zhong t P_zhong represents the total electricity volume of the medium- and long-term contracts at time t. t P_riqian represents the medium- to long-term contract weighted electricity price at time t. t The market-day clearing price at time t; Q_shishi t P_shishi represents the actual power generation at time t. t Q_riqian represents the real-time market clearing price at time t. t Q_short represents the cleared power data reported at time t. t α represents the short-term power data at acquisition time t; t This refers to the adjustment ratio data reported at time t.

[0105] Finally, the reward function value J for day d is obtained based on the difference between the deviation recovery loss and the settlement revenue on day d:

[0106] J = J jiesuan -J chaoe .

[0107] Furthermore, step five of this embodiment obtains the optimal adjustment ratio based on a multi-agent deep reinforcement learning model. This process involves pre-setting the number of training iterations for the multi-agent deep reinforcement learning model, saving the corresponding model after each training iteration, inputting validation data into the models saved after each training iteration, calculating the returns of each model, and selecting the model with the highest return as the optimal model, thereby obtaining the optimal adjustment ratio. In this embodiment, a single training iteration uses six months of data as training data, the most recent N days of data for the target date as the validation set, and the data at point T on a single day as the sequence length for training. The number of training steps is set to M*T steps, and a total of 200 training iterations are performed. The models trained each time are saved. Then, the validation set data is sequentially imported into the saved models, the validation set returns of each model are calculated, and the model with the highest return is selected as the optimal model.

[0108] Finally, the data for the target date is fed into the optimal model to obtain the final adjustment ratio for the target date, such as... Figure 3 As shown, multiplying it by the short-term power yields the final application strategy.

[0109] Furthermore, such as Figure 2 As shown, this application also discloses a power parameter auxiliary model decision-making device based on a multi-agent system, the device comprising:

[0110] The sample data acquisition module is used to acquire sample data of the electricity market corresponding to the collection time t on day d and summarize them into a sample dataset, and preprocess the sample dataset; the sample dataset includes at least day-ahead price, real-time price, market day-ahead clearing price, market real-time clearing price, clearing power data, short-term power data, adjustment ratio data and environmental parameters; where d = 1, 2, ..., D, and D is the total number of days in the sample data;

[0111] The adjustment range determination module is used to analyze the existing adjustment ratio data in the sample dataset to obtain the adjustment ratio range;

[0112] The feature data augmentation module is used to calculate the difference between the day-ahead price and the real-time price in the electricity market based on the sample dataset, and to process the difference between the day-ahead price and the real-time price into the original feature dataset through hot encoding. Then, the original feature dataset is augmented by window sliding processing to obtain the augmented feature dataset.

[0113] The model building module is used to build a multi-agent deep reinforcement learning model with settlement income and profit recovery loss as reward functions, using sample datasets, adjusted scale ranges and expanded feature datasets as input data.

[0114] The application strategy determination module is used to perform multiple trainings based on a multi-agent deep reinforcement learning model, select the best multi-agent deep reinforcement learning model, obtain the optimal adjustment ratio based on the best multi-agent deep reinforcement learning model, and obtain the optimal application parameters based on the optimal adjustment ratio and short-term power.

[0115] The apparatus provided in this application embodiment can achieve... Figure 1 To avoid repetition, the various processes implemented in the method embodiments will not be described again here.

[0116] like Figure 4 As shown in the illustration, this application also provides an electronic device, including a processor and a memory, and a program or instructions stored in the memory and executable on the processor, which, when executed by the processor, implement as follows: Figure 1 The various processes of the method embodiments shown are all capable of achieving the same technical effect, and will not be described again here to avoid repetition.

[0117] This application embodiment also provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the above-described functionality. Figure 1 The various processes described in the embodiments of the method described herein can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0118] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described... Figure 1 The various processes described in the embodiments of the method described herein can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0119] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0120] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another device, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0122] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0123] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0124] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0125] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a device (which may be a terminal or platform, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0126] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A power parameter-assisted model decision-making method based on a multi-agent system, characterized in that, Includes the following steps: Step 1: Obtain sample data of the electricity market corresponding to the t-th collection time on day d, summarize them into a sample dataset, and preprocess the sample dataset; the sample dataset includes at least the day-ahead price, real-time price, market day-ahead clearing price, market real-time clearing price, cleared power data, short-term power data, adjustment ratio data, and environmental parameters; wherein , D This represents the total number of days within the sample data. Step 2: Analyze the existing adjustment ratio data in the sample dataset to obtain the adjustment ratio range; Step 3: Calculate the difference between the day-ahead price and the real-time price in the electricity market based on the sample dataset, and process the difference between the day-ahead price and the real-time price into the original feature dataset through hot coding. Then, expand the original feature dataset by window sliding processing to obtain the expanded feature dataset. Step 4: Using the sample dataset, the adjusted scale range, and the expanded feature dataset as input data, train a multi-agent deep reinforcement learning model with settlement revenue and profit recovery loss as reward functions; The steps involved in establishing a multi-agent deep reinforcement learning model, using settlement of income and recovery of losses as reward functions, include: Obtain the actual power generation, cleared power data, and real-time market clearing price corresponding to the data collection time t on day d; The deviation range of the cleared power data at the corresponding time is calculated based on the cleared power data; Determine whether the actual power generation at time t on day d is within the deviation range of the cleared power data: if the actual power generation is within the deviation range of the cleared power data, then set the corresponding deviation recovery loss to 0; if the actual power generation is not within the deviation range of the cleared power data, then calculate the deviation recovery loss at time t. The deviation recovery loss on day d is calculated based on the deviation recovery loss corresponding to different acquisition times. The settlement revenue at time t on day d is calculated based on the actual power generation, cleared power data, day-ahead clearing price, and real-time clearing price of the market at time t during day d. The settlement revenue for day d is calculated based on the settlement revenue collected at time t on day d. The reward function value for day d is obtained based on the difference between the deviation recovery loss and the settlement revenue on day d. Step 5: Perform multiple training sessions based on the multi-agent deep reinforcement learning model, select the best multi-agent deep reinforcement learning model, obtain the optimal adjustment ratio based on the best multi-agent deep reinforcement learning model, and obtain the optimal application parameters based on the optimal adjustment ratio and short-term power.

2. The power parameter-assisted model decision-making method according to claim 1, characterized in that, In step one, when obtaining sample data of the electricity market, the data is collected in days and each day is divided into several equally spaced collection times. Sample data is collected for each collection time.

3. The power parameter-assisted model decision-making method according to claim 2, characterized in that, In step one, a data normalization method is used to preprocess the sample dataset: First, obtain the maximum and minimum values ​​of different sample data in the sample dataset, and then classify the sample data according to different times of the same day to obtain the sample units for day d. The difference between the sample data at time t is obtained based on the maximum and minimum values ​​of the corresponding sample data. Based on the sample unit of day d, obtain the sample data at time t in day d, and obtain the normalized sample data corresponding to time t in day d based on the difference between the sample data at time t in day d and the sample data corresponding to time t in day d.

4. The power parameter-assisted model decision-making method according to claim 1, characterized in that, The step in step two, which involves analyzing the existing adjustment ratio data in the sample dataset to obtain the adjustment ratio range, includes: Convert the cleared power data corresponding to the collection time t on day d in the sample dataset into power system output data; Calculate the adjustment ratio between the power system output data and the short-term power data at the time of data collection on day d at time t. The maximum and minimum values ​​of the adjustment ratio are obtained by statistical analysis of the sample dataset using statistical methods, and the maximum and minimum values ​​of the adjustment ratio are used as the range of the adjustment ratio. The probability distribution of the adjustment ratio at each collection time in the historical days is calculated using a probability density curve function.

5. The power parameter-assisted model decision-making method according to claim 1, characterized in that, The step three, which processes the difference between the day-to-day price and the real-time price into feature data using a hot coding method, includes: Obtain the day-ahead price and real-time price corresponding to the data collection time t on day d; Calculate the difference between the day-ahead price and the real-time price at time t on day d; The difference between the previous day's price and the real-time price at time t on day d is compared with the threshold 0 to determine the direction of the price difference: when the difference between the previous day's price and the real-time price is less than 0, the direction of the price difference is -1; when the difference between the previous day's price and the real-time price is equal to 0, the direction of the price difference is 0; when the difference between the previous day's price and the real-time price is greater than 0, the direction of the price difference is 1. The obtained price difference directions are summarized to obtain characteristic data.

6. The power parameter-assisted model decision-making method according to claim 5, characterized in that, The step of expanding the original feature dataset to obtain the expanded feature dataset in step three includes: Set the window width of the time window W and sliding step size The feature data of the original feature dataset are arranged in chronological order to obtain the sorted original feature dataset. The starting positions of the sorted original feature dataset are arranged according to the window width of the time window. W Move forward one sliding step at a time Then, the feature data at the time window position is set to 0 to obtain new feature data.

7. The power parameter-assisted model decision-making method according to claim 1, characterized in that, The method for calculating the deviation recovery loss includes: ； in J t The deviation recovery loss is the value collected at time t. T This represents the total number of data collection moments. This is the market-permissible deviation ratio coefficient; Q_zhong t Let t represent the total electricity volume of the medium- and long-term contracts at time t. P_zhong t The medium- to long-term contract weighted electricity price at time t; P_riqian t The market-day clearing price at time t; Pzr t The weighted average electricity price for the medium- and long-term periods and the day-ahead period at time t; Q_shishi t This represents the actual power generation at time t. P_shishi t The real-time market clearing price at time t; Q_riqian t This refers to the cleared electricity data reported at time t. Q_short t This refers to short-term power data collected at time t. This refers to the adjustment ratio data reported at time t.

8. The power parameter-assisted model decision-making method according to claim 1, characterized in that, The method for calculating the settlement revenue includes: ； in J jiesuan To summarize and calculate income; T This represents the total number of data collection moments in the input model. J_jiesuan t This represents the settlement revenue at time t. Q_zhong t Let t represent the total electricity volume of the medium- and long-term contracts at time t. P_zhong t The medium- to long-term contract weighted electricity price at time t; P_riqian t The market-day clearing price at time t; Q_shishi t This represents the actual power generation at time t. P_shishi t The real-time market clearing price at time t; Q_riqian t This refers to the cleared electricity data reported at time t. Q_short t This refers to short-term power data collected at time t. This refers to the adjustment ratio data reported at time t.

9. A power parameter auxiliary model decision-making device based on a multi-agent system, characterized in that, The device includes: The sample data acquisition module is used to acquire sample data of the electricity market corresponding to the t-th collection time on day d and summarize them into a sample dataset, and to preprocess the sample dataset; the sample dataset includes at least day-ahead price, real-time price, market day-ahead clearing price, market real-time clearing price, cleared power volume data, short-term power data, adjustment ratio data, and environmental parameters; wherein D represents the total number of days in the sample data; The adjustment range determination module is used to analyze the existing adjustment ratio data in the sample dataset to obtain the adjustment ratio range; The feature data augmentation module is used to calculate the difference between the day-ahead price and the real-time price in the electricity market based on the sample dataset, and to process the difference between the day-ahead price and the real-time price into the original feature dataset through hot encoding. Then, the original feature dataset is augmented by window sliding processing to obtain the augmented feature dataset. The model building module is used to build a multi-agent deep reinforcement learning model with settlement income and profit recovery loss as reward functions, using sample datasets, adjusted scale ranges and expanded feature datasets as input data. The steps involved in establishing a multi-agent deep reinforcement learning model, using settlement of income and recovery of losses as reward functions, include: Obtain the actual power generation, cleared power data, and real-time market clearing price corresponding to the data collection time t on day d; The deviation range of the cleared power data at the corresponding time is calculated based on the cleared power data; Determine whether the actual power generation at time t on day d is within the deviation range of the cleared power data: if the actual power generation is within the deviation range of the cleared power data, then set the corresponding deviation recovery loss to 0; if the actual power generation is not within the deviation range of the cleared power data, then calculate the deviation recovery loss at time t. The deviation recovery loss on day d is calculated based on the deviation recovery loss corresponding to different acquisition times. The settlement revenue at time t on day d is calculated based on the actual power generation, cleared power data, day-ahead clearing price, and real-time clearing price of the market at time t during day d. The settlement revenue for day d is calculated based on the settlement revenue collected at time t on day d. The reward function value for day d is obtained based on the difference between the deviation recovery loss and the settlement revenue on day d. The application strategy determination module is used to perform multiple trainings based on a multi-agent deep reinforcement learning model, select the best multi-agent deep reinforcement learning model, obtain the optimal adjustment ratio based on the best multi-agent deep reinforcement learning model, and obtain the optimal application parameters based on the optimal adjustment ratio and short-term power.

Images