A method, device, equipment and medium for electric energy control

Abstract

The present application discloses a power control method, device, equipment and medium, relating to the field of power technology. This solution takes into account the state of the energy storage system, the electricity prices during peak, valley and off-peak periods, and the fluctuations and changes in the production load of users within a historical period, that is, fully considers the parameter changes of the energy storage system under various random scenarios; based on the above three parameters, combined with the electricity energy distribution scheme and decision-making moment in the historical period, the cumulative cost value within the historical period is determined, thereby using the historical cost value as the iterative learning basis for selecting the electricity energy distribution scheme in the future period, and finally determining the target electricity energy distribution scheme to be executed in the future period among the alternative electricity energy distribution schemes in the preset strategy space, avoiding the use of fixed parameters for charge and discharge power control; compared with the expert experience method, this solution has stronger optimal strategy adaptability to various random scenarios and improves the system performance.

Description

Technical Field

[0001] The present application relates to the field of power technology, and particularly to a method, device, equipment and medium for electric energy control. Background Art

[0002] The charge-discharge control for peak-valley arbitrage of industrial and commercial energy storage is a management strategy for energy storage systems that realizes economic benefits by utilizing the peak-valley electricity price difference in the power market. Its core principle is to charge the energy storage system during the valley period with lower electricity prices (such as at night or during non-peak electricity consumption periods), and release the stored electric energy during the peak period with higher electricity prices (such as during the day or peak electricity consumption periods) to supply the load, thereby reducing the electricity cost or obtaining arbitrage benefits through the electricity price difference.

[0003] Currently, the charge-discharge control for peak-valley arbitrage of industrial and commercial energy storage mainly relies on the expert experience control method. When alarms occur at the Energy Management System (EMS), Battery Management System (BMS), and Power Conversion System (PCS) ends, this method uses preset fixed parameters for charge-discharge power control, lacking adaptability to the optimal strategies for various random scenarios. In addition, the control instructions issued by the cloud (such as the principle of valley charge and peak discharge) also need to be manually set. In the long run, this decision-making method is not automated and intelligent enough to meet the requirements of dynamic market demands and system operation optimization.

[0004] In view of the above, how to solve the problems of insufficient adaptability of the expert experience method and lack of automation and intelligence in the current charge-discharge control for peak-valley arbitrage of industrial and commercial energy storage is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of the present application is to provide a method, device, equipment and medium for electric energy control to solve the problems of insufficient adaptability of the expert experience method and lack of automation and intelligence in the current charge-discharge control for peak-valley arbitrage of industrial and commercial energy storage.

[0006] To solve the above technical problems, the present application provides an electric energy control method, including:

[0007] Obtain the electricity price information, energy storage system status information, and production load information of the user within the historical period;

[0008] Determine the electric energy distribution plan and the corresponding decision-making moment within the historical period according to the electricity price information, energy storage system status information, and production load information; wherein, the decision-making moment represents the moment of electricity price change and / or the moment when an alarm occurs in the energy storage system within the historical period;

[0009] Determine the cumulative cost value within the historical period according to the electricity price information, energy storage system status information, production load information, power distribution plan, and the corresponding decision-making moment.

[0010] Determine the target power distribution plan from the alternative power distribution plans in the preset strategy space according to the cumulative cost value, so as to execute the target power distribution plan in the future period.

[0011] On the one hand, obtain the production load information of the user, including:

[0012] Obtain the historical production load independent variables of the user; among them, the historical production load independent variables at least include historical temperature, historical production plan, and historical electricity price information.

[0013] Construct a production load prediction model according to the historical production load independent variables and the linear regression algorithm.

[0014] Obtain the target production load independent variables of the user within the historical period.

[0015] Input the target production load independent variables into the production load prediction model to determine the production load information of the user within the historical period.

[0016] On the other hand, determine the cumulative cost value within the historical period according to the electricity price information, energy storage system status information, production load information, power distribution plan, and the corresponding decision-making moment, including:

[0017] Determine the cost coefficients; among them, the cost coefficients include the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, and the maintenance downtime cost coefficient of the energy storage system to be maintained.

[0018] Divide the historical period into multiple historical decision-making cycles according to the decision-making moment.

[0019] Determine the power distribution plan corresponding to each historical decision-making cycle, and determine the power distribution type of the power distribution plan; among them, the power distribution type includes the charging process and the discharging process.

[0020] Determine the cost value within each historical decision-making cycle respectively according to the cost coefficients and the power distribution types of the power distribution plans.

[0021] Sum up the cost values within each historical decision-making cycle to obtain the cumulative cost value within the historical period.

[0022] On the other hand, determine the cost value within each historical decision-making cycle respectively according to the cost coefficients and the power distribution types of the power distribution plans, including:

[0023] When the power distribution type of the power distribution plan is the discharging process, determine the corresponding historical decision period as the discharging decision period;

[0024] Determine the discharging weight coefficient, and determine the alarm level, electricity price information, and production load information of the energy storage system during the discharging decision period;

[0025] According to the discharging weight coefficient, the alarm level of the energy storage system during the discharging decision period, and the electricity price information, determine the cumulative discharging cost during the discharging decision period.

[0026] On the other hand, according to the discharging weight coefficient, the alarm level of the energy storage system during the discharging decision period, and the electricity price information, determine the cumulative discharging cost during the discharging decision period, including:

[0027] Judge whether the alarm level of the energy storage system during the discharging decision period is greater than the first threshold;

[0028] If not, then according to the discharging weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharging reward coefficient per unit time of the energy storage system, the discharging decision period, and the power distribution plan, alarm level, electricity price information, and production load information of the energy storage system during the discharging decision period, determine the cumulative discharging cost during the discharging decision period;

[0029] If so, determine the first moment of the alarm of the energy storage system during the discharging decision period;

[0030] Divide the discharging decision period into a first normal sub-period and a first alarm sub-period according to the first moment;

[0031] According to the discharging weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharging reward coefficient per unit time of the energy storage system, the first normal sub-period, and the power distribution plan, alarm level, electricity price information, and production load information of the energy storage system during the first normal sub-period, determine the discharging cost during the first normal sub-period;

[0032] According to the discharging weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharging reward coefficient per unit time of the energy storage system, the first alarm sub-period, and the power distribution plan, alarm level, electricity price information, and production load information of the energy storage system during the first alarm sub-period, determine the discharging cost during the first alarm sub-period;

[0033] Sum up the discharging cost during the first normal sub-period and the discharging cost during the first alarm sub-period to determine the cumulative discharging cost during the discharging decision period.

[0034] On the other hand, before dividing the discharging decision period into a first normal sub-period and a first alarm sub-period according to the first moment, after determining the first moment of the alarm of the energy storage system during the discharging decision period, it further includes:

[0035] Determine whether the energy storage system needs to be shut down for maintenance after the first moment within the discharge decision cycle;

[0036] If not, enter the step of dividing the discharge decision cycle into a first normal sub-cycle and a first warning sub-cycle according to the first moment;

[0037] If so, determine the second moment when the energy storage system needs to be shut down for maintenance within the discharge decision cycle;

[0038] According to the first moment and the second moment, divide the discharge decision cycle into a second normal sub-cycle, a second warning sub-cycle and a maintenance shutdown cycle;

[0039] According to the discharge weight coefficient, the warning cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the second normal sub-cycle and the power distribution scheme within the second normal sub-cycle, the warning level of the energy storage system, the electricity price information and the production load information, determine the discharge cost within the second normal sub-cycle;

[0040] According to the discharge weight coefficient, the warning cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the maintenance shutdown cost coefficient of the energy storage system, the second warning sub-cycle and the power distribution scheme within the second warning sub-cycle, the warning level of the energy storage system, the electricity price information and the production load information, determine the discharge cost within the second warning sub-cycle;

[0041] Sum up the discharge cost within the second normal sub-cycle and the discharge cost within the second warning sub-cycle to determine the cumulative discharge cost within the discharge decision cycle.

[0042] On the other hand, according to the cost coefficient and the power distribution type of each power distribution scheme, determine the cost values within each historical decision cycle, including:

[0043] When the power distribution type of the power distribution scheme is the charging process, determine the corresponding historical decision cycle as the charging decision cycle;

[0044] Determine the charging weight coefficient, and determine the warning level of the energy storage system, the electricity price information and the production load information within the charging decision cycle;

[0045] According to the charging weight coefficient, the warning level of the energy storage system and the electricity price information within the charging decision cycle, determine the cumulative charging cost within the charging decision cycle.

[0046] On the other hand, according to the charging weight coefficient, the warning level of the energy storage system and the electricity price information within the charging decision cycle, determine the cumulative charging cost within the charging decision cycle, including:

[0047] Judge whether the warning level of the energy storage system within the charging decision cycle is greater than the second threshold;

[0048] Otherwise, determine the cumulative charging cost within the charging decision period according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the alarm cost coefficient per unit time of the energy storage system, the charging decision period, the power distribution plan within the charging decision period, the alarm level of the energy storage system, the electricity price information, and the production load information.

[0049] If so, determine the first moment of the alarm of the energy storage system within the charging decision period.

[0050] Divide the charging decision period into a first normal sub-period and a first alarm sub-period according to the first moment.

[0051] Determine the charging cost within the first normal sub-period according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the alarm cost coefficient per unit time of the energy storage system, the first normal sub-period, the power distribution plan within the first normal sub-period, the alarm level of the energy storage system, the electricity price information, and the production load information.

[0052] Determine the charging cost within the first alarm sub-period according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the alarm cost coefficient per unit time of the energy storage system, the first alarm sub-period, the power distribution plan within the first alarm sub-period, the alarm level of the energy storage system, the electricity price information, and the production load information.

[0053] Sum up the charging cost within the first normal sub-period and the charging cost within the first alarm sub-period to determine the cumulative charging cost within the charging decision period.

[0054] On the other hand, before dividing the charging decision period into a first normal sub-period and a first alarm sub-period according to the first moment, after determining the first moment of the alarm of the energy storage system within the charging decision period, it further includes:

[0055] Judge whether the energy storage system needs to be shut down for maintenance after the first moment within the charging decision period.

[0056] If not, enter the step of dividing the charging decision period into a first normal sub-period and a first alarm sub-period according to the first moment.

[0057] If so, determine the second moment when the energy storage system needs to be shut down for maintenance within the charging decision period.

[0058] Divide the charging decision period into a second normal sub-period, a second alarm sub-period, and a maintenance shutdown period according to the first moment and the second moment.

[0059] Determine the charging cost within the second normal sub - cycle based on the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the second normal sub - cycle and the power distribution plan within the second normal sub - cycle, the warning level of the energy storage system, the electricity price information, and the production load information;

[0060] Determine the charging cost within the second warning sub - cycle based on the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the maintenance - waiting shutdown cost coefficient of the energy storage system, the second warning sub - cycle and the power distribution plan within the second warning sub - cycle, the warning level of the energy storage system, the electricity price information, and the production load information;

[0061] Sum up the charging cost within the second normal sub - cycle and the charging cost within the second warning sub - cycle to determine the cumulative charging cost within the charging decision cycle.

[0062] On the other hand, determine the target power distribution plan from each candidate power distribution plan in the preset policy space according to the cumulative cost value, including:

[0063] Determine the reinforcement learning optimization objective; where the reinforcement learning optimization objective is to determine the target power distribution plan from each candidate power distribution plan in the preset policy space to minimize the expected average cost per unit time of the system over an infinite time interval;

[0064] Construct a reinforcement learning model according to the reinforcement learning optimization objective, the cumulative cost value, and each candidate power distribution plan in the preset policy space;

[0065] Solve the reinforcement learning model to obtain the target power distribution plan.

[0066] On the other hand, solving the reinforcement learning model includes:

[0067] Initialize the Q - value table and the average cost, and set the learning step decay factor, the initial temperature of simulated annealing, the cooling coefficient, and the maximum number of iterations; where the Q - value table contains the Q - values of multiple state - action pairs under the average criterion; the state is the joint state composed of electricity price information, energy storage system state information, and production load information, and the action is the candidate power distribution plan;

[0068] Read the current state at the current decision moment;

[0069] Determine the current temperature according to the initial temperature of simulated annealing, the cooling coefficient, and the current number of iterations, and determine the exploration probability according to the current temperature;

[0070] Generate a random number and determine whether the random number is less than the exploration probability;

[0071] If the random number is less than the exploration probability, randomly select an action from the Q-value table;

[0072] If the random number is not less than the exploration probability, select the action with the minimum cost corresponding to the current state from the Q-value table;

[0073] Execute the action and determine the cost of the action;

[0074] Calculate the difference result based on the cost of the action, the average cost, the minimum Q-value of the next state, and the Q-value corresponding to the current state;

[0075] Update the Q-value corresponding to the current state according to the Q-value corresponding to the current state, the learning step decay factor, and the difference result;

[0076] Update the average cost and the learning step decay factor, and determine whether the current iteration number reaches the maximum iteration number and the Q-value table meets the convergence condition;

[0077] If it is determined that the current iteration number does not reach the maximum iteration number, and / or the Q-value table does not meet the convergence condition, then use the next decision moment as the current decision moment, and return to the step of reading the current state at the current decision moment;

[0078] If it is determined that the current iteration number reaches the maximum iteration number and the Q-value table meets the convergence condition, then confirm that the Q-value table converges;

[0079] Generate a target power distribution plan according to the converged Q-value table.

[0080] On the other hand, generating a target power distribution plan according to the converged Q-value table includes:

[0081] Traverse all states in the converged Q-value table;

[0082] Select the action with the minimum cost in each state based on the converged Q-value table to obtain an action set;

[0083] Determine the action set as the target power distribution plan.

[0084] To solve the above technical problems, the present application also provides a power control device, including:

[0085] An acquisition module for acquiring electricity price information, energy storage system status information, and user's production load information within a historical period;

[0086] A first determination module for determining a power distribution plan and corresponding decision moments within a historical period according to the electricity price information, energy storage system status information, and production load information; wherein, the decision moment represents the moment when the electricity price changes within the historical period and / or the moment when the energy storage system generates an alarm;

[0087] A second determination module, configured to determine the cumulative cost value within a historical period according to the electricity price information, the energy storage system status information, the production load information, the power distribution plan, and the corresponding decision moment.

[0088] A third determination module, configured to determine a target power distribution plan from the alternative power distribution plans in the preset policy space according to the cumulative cost value, so as to execute the target power distribution plan in a future period.

[0089] To solve the above technical problems, the present application further provides an electric energy control device, including:

[0090] A memory, configured to store a computer program;

[0091] A processor, configured to implement the steps of the above-mentioned electric energy control method when executing the computer program.

[0092] To solve the above technical problems, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned electric energy control method are implemented.

[0093] The electric energy control method provided by the present application includes: obtaining the electricity price information, the energy storage system status information, and the production load information of the user within a historical period; determining the power distribution plan and the corresponding decision moment within the historical period according to the electricity price information, the energy storage system status information, and the production load information, where the decision moment represents the moment when the electricity price changes and / or the moment when the energy storage system generates an alarm within the historical period; determining the cumulative cost value within the historical period according to the electricity price information, the energy storage system status information, the production load information, the power distribution plan, and the corresponding decision moment; determining a target power distribution plan from the alternative power distribution plans in the preset policy space according to the cumulative cost value, so as to execute the target power distribution plan in a future period. It can be seen from this that this solution takes into account the state of the energy storage system, the electricity price during peak, valley, and flat periods, and the fluctuations and changes in the user's production load within the historical period, that is, it fully considers the changes in the energy storage system parameters under various random scenarios; based on the above three parameters, combined with the power distribution plan and the decision moment of the historical period, the cumulative cost value within the historical period is determined, so as to use the historical cost value as the iterative learning basis for selecting the power distribution plan in the future period, and finally determine the target power distribution plan to be executed in the future period from the alternative power distribution plans in the preset policy space, avoiding the use of fixed parameters for charge and discharge power control; compared with the expert experience method, this solution has stronger optimal strategy adaptability to various random scenarios and improves the system performance.

[0094] In addition, the present application further provides an electric energy control device, equipment, and medium, and the effects are the same as above. Description of the Drawings

[0095] To more clearly illustrate the embodiments of the present application, the accompanying drawings required for use in the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts.

[0096] Figure 1 It is a schematic diagram of the industrial and commercial energy storage peak-valley arbitrage system provided by the embodiments of the present application;

[0097] Figure 2 It is a flowchart of a power control method provided by the embodiments of the present application;

[0098] Figure 3 It is the principle diagram of the industrial and commercial energy storage peak-valley arbitrage provided by the embodiments of the present application;

[0099] Figure 4 It is a schematic diagram of a power control device provided by the embodiments of the present application;

[0100] Figure 5 It is a schematic diagram of a power control device provided by the embodiments of the present application. Detailed implementation manners

[0101] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0102] The core of the present application is to provide a power control method, device, equipment and medium to solve the problems of insufficient adaptability of the expert experience method and lack of automation and intelligence in the charge and discharge control of current industrial and commercial energy storage peak-valley arbitrage.

[0103] To enable those skilled in the art to better understand the solution of the present application, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0104] Figure 1 It is a schematic diagram of the industrial and commercial energy storage peak-valley arbitrage system provided by the embodiments of the present application. In the power consumption scenario of energy storage devices in the new energy storage field, the cloud platform issues a charge and discharge allocation strategy, and the energy storage system distributes according to the charge and discharge weights to assist users in meeting their production power consumption needs for peak-valley arbitrage. As Figure 1As shown in the figure, the industrial and commercial energy storage peak-valley arbitrage system consists of the user-side electricity load, the energy storage system, and the cloud platform. To solve the problems of insufficient adaptability of the expert experience method and lack of automation and intelligence in the charge and discharge control of current industrial and commercial energy storage peak-valley arbitrage, based on the existing hardware structure of the industrial and commercial energy storage peak-valley arbitrage system, this application proposes a power control method.

[0105] Figure 2 It is a flowchart of a power control method provided by an embodiment of this application. As Figure 2 shown, the method includes:

[0106] S10: Obtain the electricity price information, the energy storage system status information, and the user's production load information within the historical period.

[0107] First, obtain the electricity price information within the historical period , including the electricity price information during peak, valley, and flat periods, and its state space is , where k is the identifier of different electricity price information. Obtain the energy storage system status information , including but not limited to the status information of devices such as EMS, BMS, PCS, communication modules, and smart meters, and its state space is , where j is the identifier of different device status information. Obtain the user's production load information , and its state space is , where n is the identifier of different production loads. At the same time, define as the joint state of the industrial and commercial energy storage peak-valley arbitrage system, and its state space is .

[0108] It should be noted that in this embodiment, there is no limit to the size of the historical period, which is determined according to the specific implementation situation.

[0109] S11: Determine the power distribution plan and the corresponding decision-making moment within the historical period according to the electricity price information, the energy storage system status information, and the production load information.

[0110] Among them, the decision-making moment represents the moment of electricity price change and / or the moment when the energy storage system generates an alarm within the historical period.

[0111] Furthermore, determine the power distribution plan and the corresponding decision-making moment within the historical period. Specifically, take the electricity price information , the energy storage system status information , and the user's production load information as the state variables for controlling the peak-valley arbitrage strategy of the reference energy storage system. Among them, to ensure that the energy storage system can better perform its work tasks and avoid shutdown, when the system shuts down during the operation of the system, the state of the energy storage system is , the decision-making center of the cloud platform immediately stops a new round of decision-making until the device runs again, where the downtime caused by the fault is . In this embodiment, the charging and discharging allocation plan for the current period issued by the cloud platform is defined as the action of the system. In addition, the decision-making moment represents the moment when the electricity price changes and / or the moment when the energy storage system generates an alarm during the historical period, and can be determined according to the time information in the electricity price information, the energy storage system status information, and the production load information.

[0112] S12: Determine the cumulative cost value within the historical period according to the electricity price information, the energy storage system status information, the production load information, the power distribution plan, and the corresponding decision-making moment.

[0113] S13: Determine the target power distribution plan from the alternative power distribution plans in the preset strategy space according to the cumulative cost value, so as to execute the target power distribution plan in the future period.

[0114] Figure 3 is the schematic diagram of peak-valley arbitrage for industrial and commercial energy storage provided by the embodiment of the present application. As Figure 3 shown, after obtaining the electricity price information, the energy storage system status information, the production load information, and determining the power distribution plan and the corresponding decision-making moment within the historical period, according to the electricity price information, the energy storage system status information, the production load information, as well as the power distribution plan and the decision-making moment within the historical period, determine the cumulative cost value within the historical period, that is, establish a mathematical model of each state information. Finally, perform strategy solving on the mathematical model, and determine the target power distribution plan from the alternative power distribution plans in the preset strategy space. The obtained target power distribution plan is used to guide the actual working process of the industrial and commercial energy storage peak-valley arbitrage system in the future period, realizing the improvement of the peak-valley arbitrage efficiency of the energy storage system.

[0115] It should be noted that the preset strategy space contains multiple pre-generated alternative power distribution plans, including charging plans and discharging plans. In this embodiment, the specific types of the alternative power distribution plans in the preset strategy space are not limited. In addition, in this embodiment, the determination process of the cumulative cost value within the historical period is not limited, and the determination method of the target power distribution plan is also not limited, which depends on the specific implementation situation.

[0116] In this embodiment, the state of the energy storage system, the electricity prices during peak, valley, and shoulder periods, and the fluctuations and changes in the user's production load within the historical period are considered, that is, the parameter changes of the energy storage system under various random scenarios are fully considered; based on the above three parameters, combined with the electricity energy distribution plan and the decision-making moment in the historical period, the cumulative cost value within the historical period is determined, so that the historical cost value is used as the iterative learning basis for selecting the electricity energy distribution plan in the future period, and finally the target electricity energy distribution plan to be executed in the future period is determined among the alternative electricity energy distribution plans in the preset strategy space, avoiding the use of fixed parameters for charge and discharge power control; compared with the expert experience method, this solution has stronger optimal strategy adaptability to various random scenarios and improves the system performance.

[0117] Based on the above embodiment, in some embodiments, the production load information of the user is obtained, including:

[0118] S101: Obtain the historical production load independent variables of the user.

[0119] Among them, the historical production load independent variables at least include historical temperature, historical production plan, and historical electricity price information.

[0120] S102: Construct a production load prediction model according to the historical production load independent variables and the linear regression algorithm.

[0121] S103: Obtain the target production load independent variables of the user within the historical period.

[0122] S104: Input the target production load independent variables into the production load prediction model to determine the production load information of the user within the historical period.

[0123] In order to accurately obtain the production load information of the user, in this embodiment, a linear regression is used to construct a mathematical model for predicting the production load of the user. Specifically, the historical production load independent variables of the user are obtained. It can be understood that the historical production load independent variables are the production load independent variables of the user during the historical production process, at least including historical temperature, historical production plan, and historical electricity price information, and may also include other information, which is not limited in this embodiment. Subsequently, a production load prediction model is constructed according to the historical production load independent variables and the linear regression algorithm, as follows:

[0124] ;

[0125] Among them, is the dependent variable, that is, the production load information; are different historical production load independent variables respectively; is the intercept; are the regression coefficients respectively, representing the influence degree of each independent variable on the dependent variable; is the error; is a positive integer.

[0126] Furthermore, obtain the target production load independent variables of the user within the historical period, including but not limited to the target temperature, target production plan, and target electricity price information. Finally, input the target production load independent variables into the production load prediction model to determine the production load information of the user within the historical period. In this way, the analysis and prediction of the user's production load information are realized.

[0127] Based on the above embodiments, in some embodiments, according to the electricity price information, energy storage system status information, production load information, power distribution plan, and the corresponding decision-making moment, determine the cumulative cost value within the historical period, including:

[0128] S111: Determine the cost coefficient;

[0129] Among them, the cost coefficient includes the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, and the maintenance downtime cost coefficient of the energy storage system to be maintained.

[0130] S112: Divide the historical period into multiple historical decision-making cycles according to the decision-making moment.

[0131] S113: Determine the power distribution plan corresponding to each historical decision-making cycle, and determine the power distribution type of the power distribution plan.

[0132] Among them, the power distribution type includes the charging process and the discharging process.

[0133] S114: According to the cost coefficient and the power distribution type of each power distribution plan, respectively determine the cost value within each historical decision-making cycle.

[0134] S115: Sum up the cost values within each historical decision-making cycle to obtain the cumulative cost value within the historical period.

[0135] In order to determine the cumulative cost value within the historical period, in this embodiment, the cost composition of the system is specifically defined: charging cost, warning cost, discharge reward, and downtime cost. At the same time, the cost coefficients corresponding to each cost are defined: the charging cost coefficient per unit time of the energy storage system , the warning cost coefficient per unit time of the energy storage system , the discharge reward coefficient per unit time of the energy storage system (usually a negative number, representing the actual peak-valley arbitrage reward), the maintenance downtime cost coefficient of the energy storage system to be maintained .

[0136] Due to the price changes of peak, valley, and flat periods within the historical period, the energy storage system status, user production load, and the actual electricity distribution plan used are also different in different cycles. Therefore, it is necessary to divide the historical period into multiple historical decision cycles according to the decision-making moment. For example, assume that a factory faces different production loads and electricity price fluctuations within a week. To optimize the use of the energy storage system, the historical period can be divided into multiple decision cycles according to the decision-making moment. For example, from Monday to Wednesday, the production load is low, and the valley period electricity price is significantly lower than the peak period, which is suitable for charging a large amount during the valley period and discharging during the peak period to reduce costs; from Thursday to Friday, the production load increases, and the peak period electricity price rises. It is necessary to adjust the strategy and increase charging during the flat period to cope with higher electricity demand.

[0137] Further determine the electricity distribution plan corresponding to each historical decision cycle and determine the electricity distribution type of the electricity distribution plan. It can be understood that the electricity distribution type includes the charging process and the discharging process. That is to say, in the electricity distribution plan corresponding to each historical decision cycle, there are charging plans and discharging plans. Finally, according to the cost coefficient and the electricity distribution type of each electricity distribution plan, determine the cost value within each historical decision cycle respectively, and sum up the cost values within each historical decision cycle to achieve the determination of the cumulative cost value within the historical period, so as to facilitate the determination of the target electricity distribution plan for the future period using the cumulative cost value.

[0138] The following uses specific embodiments to illustrate the determination process of the cost value within the historical decision cycle:

[0139] (1) Discharging process;

[0140] Based on the above embodiments, in some embodiments, according to the cost coefficient and the electricity distribution type of each electricity distribution plan, determine the cost value within each historical decision cycle respectively, including:

[0141] S120: When the electricity distribution type of the electricity distribution plan is the discharging process, determine the corresponding historical decision cycle as the discharging decision cycle;

[0142] S121: Determine the discharging weight coefficient, and determine the energy storage system alarm level, electricity price information, and production load information within the discharging decision cycle;

[0143] S122: According to the discharging weight coefficient, the energy storage system alarm level, and the electricity price information within the discharging decision cycle, determine the cumulative discharging cost within the discharging decision cycle.

[0144] Specifically, when the power distribution type of the power distribution plan is the discharging process, the corresponding historical decision period is determined as the discharging decision period, and within the discharging decision period, the discharging process is executed. Subsequently, the discharging weight coefficient is determined, and the alarm level, electricity price information, and production load information of the energy storage system within the discharging decision period are determined. Finally, according to the discharging weight coefficient, the alarm level of the energy storage system, and the electricity price information within the discharging decision period, the cumulative discharging cost within the discharging decision period is determined.

[0145] It should be noted that the cumulative cost is expressed as , specifically representing the state at the decision moment and jumping to the next decision moment under the action accumulated cost. Meanwhile, under normal operation, the cumulative discharging cost within the discharging decision period needs to be calculated in different cases, as follows:

[0146] (1) Alarm of the energy storage system;

[0147] Specifically, according to the discharging weight coefficient, the alarm level of the energy storage system, and the electricity price information within the discharging decision period, the cumulative discharging cost within the discharging decision period is determined, including:

[0148] S123: Determine whether the alarm level of the energy storage system within the discharging decision period is greater than the first threshold; if not, proceed to step S124; if so, proceed to step S125;

[0149] S124: According to the discharging weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharging reward coefficient per unit time of the energy storage system, the discharging decision period, and the power distribution plan, the alarm level, the electricity price information, and the production load information within the discharging decision period, determine the cumulative discharging cost within the discharging decision period;

[0150] S125: Determine the first moment of the alarm of the energy storage system within the discharging decision period;

[0151] S126: Divide the discharging decision period into a first normal sub-period and a first alarm sub-period according to the first moment;

[0152] S127: According to the discharging weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharging reward coefficient per unit time of the energy storage system, the first normal sub-period, and the power distribution plan, the alarm level, the electricity price information, and the production load information within the first normal sub-period, determine the discharging cost within the first normal sub-period;

[0153] S128: Determine the discharge cost within the first warning sub - cycle based on the discharge weight coefficient, the warning cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the first warning sub - cycle, the power distribution plan within the first warning sub - cycle, the warning level of the energy storage system, the electricity price information, and the production load information.

[0154] S129: Add the discharge cost within the first normal sub - cycle and the discharge cost within the first warning sub - cycle to determine the cumulative discharge cost within the discharge decision cycle.

[0155] Specifically, first determine whether the warning level of the energy storage system within the discharge decision cycle is greater than the first threshold. It should be noted that in this embodiment, there is no limit on the classification of the warning level of the energy storage system, nor on the size of the first threshold. For example, the warning can be classified into level 1 warning, level 2 warning, and level 3 warning according to the severity of the warning, and the first threshold is set to 2. Then, when the warning level of the energy storage system is level 1 warning or level 2 warning, it is not greater than the first threshold, and when the warning level of the energy storage system is level 3 warning, it is greater than the first threshold.

[0156] If it is confirmed that the warning level of the energy storage system within the discharge decision cycle is not greater than the first threshold, it is confirmed that the energy storage system discharges normally within the discharge decision cycle. Specifically, determine the cumulative discharge cost within the discharge decision cycle according to the discharge weight coefficient, the warning cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the discharge decision cycle, the power distribution plan within the discharge decision cycle, the warning level of the energy storage system, the electricity price information, and the production load information. The formula is as follows:

[0157] ;

[0158] Wherein, is the cumulative discharge cost within the discharge decision cycle, is the discharge weight coefficient, is the power distribution plan within the discharge decision cycle, is the discharge decision cycle, is the warning level of the energy storage system within the discharge decision cycle, is the electricity price information within the discharge decision cycle, is the production load information within the discharge decision cycle, is the warning cost coefficient per unit time of the energy storage system, is the discharge reward coefficient per unit time of the energy storage system.

[0159] If it is confirmed that the warning level of the energy storage system within the discharge decision cycle is greater than the first threshold, it is confirmed that there is an abnormal warning in the energy storage system within the discharge decision cycle. It is also necessary to determine the first moment of the warning of the energy storage system within the discharge decision cycle, and divide the discharge decision cycle according to the first moment Divided into a first normal sub - cycle and a first alarm sub - cycle . During the first normal sub - cycle, the energy storage system does not alarm. During the first alarm sub - cycle, the energy storage system alarms. Subsequently, according to the discharge weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the first normal sub - cycle and the power distribution scheme within the first normal sub - cycle, the alarm level of the energy storage system, the electricity price information, and the production load information, determine the discharge cost within the first normal sub - cycle. The formula is as follows:

[0160]

[0161] where is the discharge cost within the first normal sub - cycle, is the discharge weight coefficient, is the power distribution scheme within the first normal sub - cycle, is the first normal sub - cycle, is the alarm level of the energy storage system within the first normal sub - cycle, is the electricity price information within the first normal sub - cycle, is the production load information within the first normal sub - cycle, is the alarm cost coefficient per unit time of the energy storage system, is the discharge reward coefficient per unit time of the energy storage system.

[0162] Meanwhile, according to the discharge weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the first alarm sub - cycle and the power distribution scheme within the first alarm sub - cycle, the alarm level of the energy storage system, the electricity price information, and the production load information, determine the discharge cost within the first alarm sub - cycle. The formula is as follows:

[0163] ;

[0164] where is the discharge cost within the first alarm sub - cycle, is the discharge weight coefficient, is the power distribution scheme within the first alarm sub - cycle, is the first alarm sub - cycle, is the alarm level of the energy storage system within the first alarm sub - cycle, is the electricity price information within the first alarm sub - cycle, is the production load information within the first alarm sub - cycle, is the alarm cost coefficient per unit time of the energy storage system, is the discharge reward coefficient per unit time of the energy storage system.

[0165] Finally, add the discharge cost during the first normal sub - period to the discharge cost during the first warning sub - period to determine the cumulative discharge cost within the discharge decision period.

[0166] (2) The energy storage system awaits maintenance and shutdown;

[0167] Specifically, before dividing the discharge decision period into the first normal sub - period and the first warning sub - period according to the first moment, after determining the first moment of the energy storage system warning within the discharge decision period, it further includes:

[0168] S130: Determine whether the energy storage system awaits maintenance and shutdown after the first moment within the discharge decision period; if not, proceed to step S126; if so, proceed to step S131.

[0169] S131: Determine the second moment when the energy storage system awaits maintenance and shutdown within the discharge decision period.

[0170] S132: Divide the discharge decision period into a second normal sub - period, a second warning sub - period, and a maintenance - awaiting shutdown period according to the first moment and the second moment.

[0171] S133: Determine the discharge cost within the second normal sub - period according to the discharge weight coefficient, the warning cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the second normal sub - period, and the power distribution plan within the second normal sub - period, the warning level of the energy storage system, the electricity price information, and the production load information.

[0172] S134: Determine the discharge cost within the second warning sub - period according to the discharge weight coefficient, the warning cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the maintenance - awaiting shutdown cost coefficient of the energy storage system, the second warning sub - period, and the power distribution plan within the second warning sub - period, the warning level of the energy storage system, the electricity price information, and the production load information.

[0173] S135: Add the discharge cost within the second normal sub - period to the discharge cost within the second warning sub - period to determine the cumulative discharge cost within the discharge decision period.

[0174] In this embodiment, after determining the first moment of the energy storage system alarm within the discharge decision period, it is also necessary to further confirm whether the energy storage system has a shutdown for operation and maintenance after a period of time. It can be understood that after the energy storage system shuts down, it will not output external electric energy and will not incur any cost. Therefore, the situation of the energy storage system shutting down for operation and maintenance needs to be taken into account. If it is confirmed that the energy storage system does not have a shutdown for operation and maintenance after the first moment within the discharge decision period, then proceed to the step of dividing the discharge decision period into a first normal sub-period and a first alarm sub-period according to the first moment.

[0175] If it is confirmed that the energy storage system has a shutdown for operation and maintenance after the first moment within the discharge decision period, then determine the second moment of the energy storage system shutdown for operation and maintenance within the discharge decision period. According to the first moment and the second moment, divide the discharge decision period into a second normal sub-period and a second alarm sub-period and a shutdown for operation and maintenance period. Since the energy storage system does not incur any cost during the shutdown for operation and maintenance, it is only necessary to determine the discharge cost within the second normal sub-period and the second alarm sub-period. That is.

[0176] Specifically, according to the discharge weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the second normal sub-period and the electric energy distribution plan within the second normal sub-period, the energy storage system alarm level, the electricity price information and the production load information, determine the discharge cost within the second normal sub-period. The formula is as follows:

[0177] ;

[0178] where is the discharge cost within the second normal sub-period, is the discharge weight coefficient, is the electric energy distribution plan within the second normal sub-period, is the second normal sub-period, is the energy storage system alarm level within the second normal sub-period, is the electricity price information within the second normal sub-period, is the production load information within the second normal sub-period, is the alarm cost coefficient per unit time of the energy storage system, is the discharge reward coefficient per unit time of the energy storage system.

[0179] Meanwhile, according to the discharge weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the cost coefficient of the energy storage system waiting for operation and maintenance shutdown, the second alarm sub-cycle and the power distribution plan within the second alarm sub-cycle, the alarm level of the energy storage system, the electricity price information and the production load information, determine the discharge cost within the second alarm sub-cycle. The formula is as follows:

[0180] ;

[0181] Among them, is the discharge cost within the second alarm sub-cycle, is the discharge weight coefficient, is the power distribution plan within the second alarm sub-cycle, is the second alarm sub-cycle, is the alarm level of the energy storage system within the second alarm sub-cycle, is the electricity price information within the second alarm sub-cycle, is the production load information within the second alarm sub-cycle, is the alarm cost coefficient per unit time of the energy storage system, is the discharge reward coefficient per unit time of the energy storage system, is the cost coefficient of the energy storage system waiting for operation and maintenance shutdown.

[0182] Finally, add the discharge cost within the second normal sub-cycle and the discharge cost within the second alarm sub-cycle to determine the cumulative discharge cost within the discharge decision cycle.

[0183] (2) Charging process;

[0184] Based on the above embodiments, in some embodiments, according to the cost coefficient and the power distribution type of each power distribution plan, determine the cost values within each historical decision cycle, including:

[0185] S140: When the power distribution type of the power distribution plan is the charging process, determine the corresponding historical decision cycle as the charging decision cycle;

[0186] S141: Determine the charging weight coefficient, and determine the alarm level of the energy storage system, the electricity price information and the production load information within the charging decision cycle;

[0187] S142: According to the charging weight coefficient, the alarm level of the energy storage system and the electricity price information within the charging decision cycle, determine the cumulative charging cost within the charging decision cycle.

[0188] Specifically, when the power distribution type of the power distribution plan is the charging process, the corresponding historical decision period is determined as the charging decision period, and within the charging decision period, the charging process is executed. Subsequently, the charging weight coefficient is determined, and the warning level of the energy storage system, electricity price information, and production load information within the charging decision period are determined. Finally, based on the charging weight coefficient, the warning level of the energy storage system, and the electricity price information within the charging decision period, the cumulative charging cost within the charging decision period is determined.

[0189] It should be noted that the cumulative cost is expressed as , which specifically characterizes the state at the decision-making moment . In the action to jump to the next decision-making moment the accumulated cost. Similarly to the discharging process, under normal operation, the cumulative charging cost within the charging decision period also needs to be calculated in different cases, as follows:

[0190] (1) Warning of the energy storage system;

[0191] Specifically, based on the charging weight coefficient, the warning level of the energy storage system, and the electricity price information within the charging decision period, the cumulative charging cost within the charging decision period is determined, including:

[0192] S143: Determine whether the warning level of the energy storage system within the charging decision period is greater than the second threshold; if not, proceed to step S144; if so, proceed to step S145.

[0193] S144: Based on the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the charging decision period, and the power distribution plan, warning level of the energy storage system, electricity price information, and production load information within the charging decision period, determine the cumulative charging cost within the charging decision period.

[0194] S145: Determine the first moment of the warning of the energy storage system within the charging decision period.

[0195] S146: Divide the charging decision period into a first normal sub-period and a first warning sub-period according to the first moment.

[0196] S147: Based on the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the first normal sub-period, and the power distribution plan, warning level of the energy storage system, electricity price information, and production load information within the first normal sub-period, determine the charging cost within the first normal sub-period.

[0197] S148: Determine the charging cost within the first warning sub - cycle based on the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the first warning sub - cycle, the power distribution plan within the first warning sub - cycle, the warning level of the energy storage system, the electricity price information, and the production load information.

[0198] S149: Add the charging cost within the first normal sub - cycle and the charging cost within the first warning sub - cycle to determine the cumulative charging cost within the charging decision cycle.

[0199] Specifically, first determine whether the warning level of the energy storage system within the charging decision cycle is greater than the second threshold. It should be noted that in this embodiment, there is no restriction on the classification of the warning level of the energy storage system, nor on the size of the second threshold. For example, it can be divided into level - 1 warning, level - 2 warning, and level - 3 warning according to the severity of the warning, and the second threshold is set to 2. Then, when the warning level of the energy storage system is level - 1 warning or level - 2 warning, it is not greater than the second threshold, and when the warning level of the energy storage system is level - 3 warning, it is greater than the second threshold.

[0200] If it is confirmed that the warning level of the energy storage system within the charging decision cycle is not greater than the second threshold, it is confirmed that the energy storage system is charging normally within the charging decision cycle. Specifically, determine the cumulative charging cost within the charging decision cycle based on the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the charging decision cycle, the power distribution plan within the charging decision cycle, the warning level of the energy storage system, the electricity price information, and the production load information. The formula is as follows:

[0201] ;

[0202] Where, is the cumulative charging cost within the charging decision cycle, is the charging weight coefficient, is the power distribution plan within the charging decision cycle, is the charging decision cycle, is the warning level of the energy storage system within the charging decision cycle, is the electricity price information within the charging decision cycle, is the production load information within the charging decision cycle, is the charging cost coefficient per unit time of the energy storage system, is the warning cost coefficient per unit time of the energy storage system.

[0203] If it is confirmed that the warning level of the energy storage system within the charging decision cycle is greater than the second threshold, it is confirmed that there is an abnormal warning in the energy storage system within the charging decision cycle. It is also necessary to determine the first moment of the warning of the energy storage system within the charging decision cycle. According to the first moment, divide the charging decision cycle Divided into a first normal sub - cycle and a first alarm sub - cycle . During the first normal sub - cycle, the energy storage system does not give an alarm. During the first alarm sub - cycle, the energy storage system gives an alarm. Subsequently, according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the alarm cost coefficient per unit time of the energy storage system, the first normal sub - cycle and the power distribution scheme within the first normal sub - cycle, the alarm level of the energy storage system, the electricity price information and the production load information, the charging cost within the first normal sub - cycle is determined. The formula is as follows:

[0204] ;

[0205] Among them, is the charging cost within the first normal sub - cycle, is the charging weight coefficient, is the power distribution scheme within the first normal sub - cycle, is the first normal sub - cycle, is the alarm level of the energy storage system within the first normal sub - cycle, is the electricity price information within the first normal sub - cycle, is the production load information within the first normal sub - cycle, is the charging cost coefficient per unit time of the energy storage system, is the alarm cost coefficient per unit time of the energy storage system.

[0206] At the same time, according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the alarm cost coefficient per unit time of the energy storage system, the first alarm sub - cycle and the power distribution scheme within the first alarm sub - cycle, the alarm level of the energy storage system, the electricity price information and the production load information, the charging cost within the first alarm sub - cycle is determined. The formula is as follows:

[0207] ;

[0208] Among them, is the charging cost within the first alarm sub - cycle, is the charging weight coefficient, is the power distribution scheme within the first alarm sub - cycle, is the first alarm sub - cycle, is the alarm level of the energy storage system within the first alarm sub - cycle, is the electricity price information within the first alarm sub - cycle, is the production load information within the first alarm sub - cycle, is the charging cost coefficient per unit time of the energy storage system, is the alarm cost coefficient per unit time of the energy storage system.

[0209] Finally, add the charging cost during the first normal sub-cycle to the charging cost during the first warning sub-cycle to determine the cumulative charging cost within the charging decision cycle.

[0210] (2) The energy storage system awaits operation and maintenance shutdown;

[0211] Specifically, before dividing the charging decision cycle into the first normal sub-cycle and the first warning sub-cycle according to the first moment, after determining the first moment of energy storage system warning within the charging decision cycle, it further includes:

[0212] S150: Determine whether the energy storage system awaits operation and maintenance shutdown after the first moment within the charging decision cycle; if not, proceed to step S146; if so, proceed to step S151.

[0213] S151: Determine the second moment when the energy storage system awaits operation and maintenance shutdown within the charging decision cycle.

[0214] S152: Divide the charging decision cycle into a second normal sub-cycle, a second warning sub-cycle, and an operation and maintenance shutdown period according to the first moment and the second moment.

[0215] S153: Determine the charging cost within the second normal sub-cycle according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the second normal sub-cycle, and the power distribution plan within the second normal sub-cycle, the warning level of the energy storage system, the electricity price information, and the production load information.

[0216] S154: Determine the charging cost within the second warning sub-cycle according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the operation and maintenance shutdown cost coefficient of the energy storage system, the second warning sub-cycle, and the power distribution plan within the second warning sub-cycle, the warning level of the energy storage system, the electricity price information, and the production load information.

[0217] S155: Add the charging cost within the second normal sub-cycle to the charging cost within the second warning sub-cycle to determine the cumulative charging cost within the charging decision cycle.

[0218] In this embodiment, after determining the first moment of the energy storage system alarm within the charging decision cycle, it is further necessary to confirm whether the energy storage system has a downtime for operation and maintenance after a period of time. It can be understood that the energy storage system will not charge after shutdown and will not incur any costs. Therefore, the situation of the energy storage system's downtime for operation and maintenance needs to be taken into account. If it is confirmed that the energy storage system does not have a downtime for operation and maintenance after the first moment within the charging decision cycle, then proceed to the step of dividing the charging decision cycle into a first normal sub-cycle and a first alarm sub-cycle according to the first moment.

[0219] If it is confirmed that the energy storage system has a downtime for operation and maintenance after the first moment within the charging decision cycle, then determine the second moment of the energy storage system's downtime for operation and maintenance within the charging decision cycle. According to the first moment and the second moment, divide the charging decision cycle into a second normal sub-cycle , a second alarm sub-cycle and a downtime for operation and maintenance cycle . Also, since no costs are incurred by the energy storage system during the downtime for operation and maintenance, it is only necessary to determine the charging costs within the second normal sub-cycle and the second alarm sub-cycle .

[0220] Specifically, according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the alarm cost coefficient per unit time of the energy storage system, the second normal sub-cycle and the power distribution plan within the second normal sub-cycle, the energy storage system alarm level, the electricity price information, and the production load information, determine the charging cost within the second normal sub-cycle. The formula is as follows:

[0221] ;

[0222] where, is the charging cost within the second normal sub-cycle, is the charging weight coefficient, is the power distribution plan within the second normal sub-cycle, is the second normal sub-cycle, is the energy storage system alarm level within the second normal sub-cycle, is the electricity price information within the second normal sub-cycle, is the production load information within the second normal sub-cycle, is the charging cost coefficient per unit time of the energy storage system, is the alarm cost coefficient per unit time of the energy storage system.

[0223] Meanwhile, according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the maintenance downtime cost coefficient of the energy storage system to be maintained, the second warning sub-cycle and the power distribution plan within the second warning sub-cycle, the warning level of the energy storage system, the electricity price information and the production load information, determine the charging cost within the second warning sub-cycle. The formula is as follows:

[0224] ;

[0225] Among them, is the charging cost within the second warning sub-cycle, is the charging weight coefficient, is the power distribution plan within the second warning sub-cycle, is the second warning sub-cycle, is the warning level of the energy storage system within the second warning sub-cycle, is the electricity price information within the second warning sub-cycle, is the production load information within the second warning sub-cycle, is the charging cost coefficient per unit time of the energy storage system, is the warning cost coefficient per unit time of the energy storage system, is the maintenance downtime cost coefficient of the energy storage system to be maintained.

[0226] Finally, add the charging cost within the second normal sub-cycle and the charging cost within the second warning sub-cycle to determine the cumulative charging cost within the charging decision cycle.

[0227] In summary, the determination of the cumulative cost value within the historical period is realized.

[0228] On the basis of the above embodiments, in some embodiments, to determine the target power distribution plan from the alternative power distribution plans in the preset policy space according to the cumulative cost value, it includes:

[0229] S161: Determine the reinforcement learning optimization goal.

[0230] Among them, the reinforcement learning optimization goal is to determine the target power distribution plan from the alternative power distribution plans in the preset policy space, so as to minimize the expected average cost per unit time of the system within an infinite time interval.

[0231] S162: Construct a reinforcement learning model according to the reinforcement learning optimization goal, the cumulative cost value and the alternative power distribution plans in the preset policy space.

[0232] S163: Solve the reinforcement learning model to obtain the target power distribution plan.

[0233] To determine the target power distribution scheme among the candidate power distribution schemes, in this embodiment, a reinforcement learning optimization objective is specifically determined, which is to determine the target power distribution scheme among the candidate power distribution schemes in the preset policy space, so as to minimize the expected average cost per unit time of the system in an infinite time interval. Further, according to the reinforcement learning optimization objective, the cumulative cost value, and the candidate power distribution schemes in the preset policy space, a reinforcement learning model is constructed, and the formula is specifically as follows:

[0234] ;

[0235] Among them, is the optimal power distribution scheme, is the candidate power distribution scheme, is the preset policy space, D is the number of time steps, is the expectation under the candidate power distribution scheme below, is the time interval, is the decision-making moment of the state at the action jumps to the next decision-making moment accumulated cost.

[0236] Finally, the above-mentioned reinforcement learning model is solved to obtain the target power distribution scheme. In this embodiment, the acquisition of the target power distribution scheme is realized by constructing and solving the reinforcement learning model, and the charge and discharge strategy control of the energy storage peak-valley arbitrage system is better optimized from the aspect of the overall stable operation of the system.

[0237] On the basis of the above embodiment, in some embodiments, solving the reinforcement learning model includes:

[0238] S170: Initialize the Q-value table and the average cost, and set the learning step size decay factor, the simulated annealing initial temperature, the cooling coefficient, and the maximum number of iterations.

[0239] Among them, the Q-value table contains the Q-values of multiple state-action pairs under the average criterion; the state is a joint state composed of electricity price information, energy storage system state information, and production load information, and the action is the candidate power distribution scheme.

[0240] S171: Read the current state at the current decision-making moment.

[0241] S172: Determine the current temperature according to the simulated annealing initial temperature, the cooling coefficient, and the current number of iterations, and determine the exploration probability according to the current temperature.

[0242] S173: Generate a random number, and determine whether the random number is less than the exploration probability; if so, go to step S174; if not, go to step S175.

[0243] S174: Randomly select an action from the Q - value table.

[0244] S175: Select the action with the minimum cost corresponding to the current state from the Q - value table.

[0245] S176: Execute the action and determine the cost of the action.

[0246] S177: Calculate the difference result according to the cost of the action, the average cost, the minimum Q - value of the next state, and the Q - value corresponding to the current state.

[0247] S178: Update the Q - value corresponding to the current state according to the Q - value corresponding to the current state, the learning step - size decay factor, and the difference result.

[0248] S179: Update the average cost and the learning step - size decay factor, and determine whether the current iteration number reaches the maximum iteration number and the Q - value table satisfies the convergence condition; if not, take the next decision moment as the current decision moment and return to step S171; if so, confirm that the Q - value table converges and enter step S180.

[0249] S180: Generate a target power distribution plan according to the converged Q - value table.

[0250] In this embodiment, the Q - learning algorithm based on simulated annealing is used to solve the strategy of the established reinforcement learning model to realize the optimization of peak - valley arbitrage control of the energy storage system.

[0251] First, initialize the Q - value table and the average cost , and set the learning step - size decay factor , the initial temperature of simulated annealing , the cooling coefficient , and the maximum iteration number . It should be noted that the Q - value table contains the Q - values of multiple state - action pairs under the average criterion; the state is a joint state composed of electricity price information, energy storage system state information, and production load information, and the action is the power distribution plan to be selected. In specific implementation, the initial value of the Q - value table can be set to 0.

[0252] Further read the current state at the current decision moment . Determine the current temperature according to the initial temperature of simulated annealing , the cooling coefficient , and the current iteration number , and determine the exploration probability according to the current temperature, as follows:

[0253] ;

[0254] ;

[0255] Wherein, is the current temperature, is the exploration probability.

[0256] Subsequently, a random number is generated in the range of 0 to 1, and it is judged whether the random number is less than the exploration probability. If it is confirmed that the random number is less than the exploration probability, an action is randomly selected from the Q-value table . If it is confirmed that the random number is not less than the exploration probability, the action with the minimum cost corresponding to the current state is selected from the Q-value table . The action is executed , and the cost of the action is determined .

[0257] Further, according to the cost of the action, the average cost, the minimum Q-value of the next state, and the Q-value corresponding to the current state, a differential result is calculated, and the formula is as follows:

[0258] ;

[0259] Wherein, is the differential result, is the cost of the action , is the average cost, is the minimum Q-value of the next state, is the Q-value corresponding to the current state.

[0260] According to the Q-value corresponding to the current state, the learning step size decay factor, and the differential result, the Q-value corresponding to the current state is updated, and the formula is as follows:

[0261] ;

[0262] Subsequently, the parameters are adjusted, and the average cost and the learning step size decay factor are updated. In this embodiment, the specific update methods for the average cost and the learning step size decay factor are not limited and are determined according to the specific implementation situation. It is judged whether the current iteration number reaches the maximum iteration number , and the Q-value table satisfies the convergence condition. If it is confirmed that the current iteration number does not reach the maximum iteration number , and / or the Q-value table does not satisfy the convergence condition, then the next decision moment is taken as the current decision moment, and the process returns to the step of reading the current state at the current decision moment. If it is confirmed that the current iteration number reaches the maximum iteration number , and the Q-value table satisfies the convergence condition, then it is confirmed that the Q-value table converges, and a target power distribution scheme is generated according to the converged Q-value table.

[0263] It should be noted that in this embodiment, there is no limitation on the specific method for generating the target power distribution scheme according to the converged Q-value table, which depends on the specific implementation situation.

[0264] In this embodiment, the Q-learning algorithm with simulated annealing is used to optimize and solve the reinforcement learning model. Compared with other algorithms, the Q-learning algorithm based on simulated annealing can evaluate the expected utility of available actions without the need for an environmental model, can effectively solve the problems of random transfer and random reward without any modification, avoid local optima at the same time, and has a fast convergence speed.

[0265] Based on the above embodiment, in some embodiments, generating the target power distribution scheme according to the converged Q-value table includes:

[0266] S181: Traverse all states in the converged Q-value table.

[0267] S182: Select the minimum-cost action in each state based on the converged Q-value table to obtain an action set.

[0268] S183: Determine the action set as the target power distribution scheme.

[0269] In order to generate the target power distribution scheme according to the converged Q-value table, in this embodiment, specifically traverse all states in the converged Q-value table, and select the minimum-cost action in each state based on the converged Q-value table. The formula is as follows:

[0270] ;

[0271] Among them, is the minimum-cost action; correspondingly, the action set is , and the action set is determined as the target power distribution scheme. During the operation of the energy storage system, call the target power distribution scheme according to the real-time state and execute the corresponding charge and discharge instructions.

[0272] In the above embodiment, the power control method is described in detail. The present application also provides an embodiment corresponding to the power control device.

[0273] Figure 4 It is a schematic diagram of a power control device provided by an embodiment of the present application. As Figure 4 shown, the device includes:

[0274] An acquisition module 10, configured to acquire electricity price information, energy storage system state information, and production load information of a user within a historical period.

[0275] The first determination module 11 is configured to determine an electric energy distribution scheme and a corresponding decision moment within a historical period according to electricity price information, energy storage system status information, and production load information; wherein, the decision moment represents the moment when the electricity price changes and / or the moment when the energy storage system generates an alarm within the historical period.

[0276] The second determination module 12 is configured to determine the cumulative cost value within a historical period according to the electricity price information, energy storage system status information, production load information, electric energy distribution scheme, and the corresponding decision moment.

[0277] The third determination module 13 is configured to determine a target electric energy distribution scheme from the alternative electric energy distribution schemes in the preset policy space according to the cumulative cost value, so as to execute the target electric energy distribution scheme in a future period.

[0278] Since the embodiments of the apparatus part correspond to the embodiments of the method part, for the embodiments of the apparatus part, please refer to the description of the embodiments of the method part, which will not be elaborated here.

[0279] Figure 5 This is a schematic diagram of an electric energy control device provided by an embodiment of the present application. As Figure 5 shown, the electric energy control device includes:

[0280] A memory 20 for storing a computer program;

[0281] A processor 21 for implementing the steps of the electric energy control method mentioned in the above embodiments when executing the computer program.

[0282] The electric energy control device provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer, or a desktop computer, etc.

[0283] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in at least one hardware form of a Digital Signal Processor (DSP), a Field-Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the Central Processing Unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a Graphics Processing Unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may further include an Artificial Intelligence (AI) processor, and the AI processor is used to process computational operations related to machine learning.

[0284] The memory 20 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 20 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201. After the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the power control method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, the data involved in the power control method.

[0285] In some embodiments, the power control device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0286] Those skilled in the art can understand that Figure 5 the structure shown in does not constitute a limitation on the power control device, and it may include more or fewer components than shown in the figure.

[0287] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps described in the above method embodiments are implemented.

[0288] It can be understood that if the method in the above embodiments is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0289] The above provides a detailed introduction to a power control method, device, equipment, and medium provided by the present application. The various embodiments in the specification are described in a progressive manner, and the key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the present application.

[0290] It should also be noted that in this specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

Claims

1. An electric energy control method, characterized in that, Including: Obtain the electricity price information, energy storage system status information, and the production load information of the user within a historical period; Determine the electricity energy distribution plan and the corresponding decision-making moment within the historical period according to the electricity price information, the energy storage system status information, and the production load information; wherein, the decision-making moment represents the moment of electricity price change and / or the moment when the energy storage system generates an alarm within the historical period; Determine the cumulative cost value within the historical period according to the electricity price information, the energy storage system status information, the production load information, the electricity energy distribution plan, and the corresponding decision-making moment; Determine the target electricity energy distribution plan from each candidate electricity energy distribution plan in the preset policy space according to the cumulative cost value, so as to execute the target electricity energy distribution plan in the future period; Wherein, determining the cumulative cost value within the historical period according to the electricity price information, the energy storage system status information, the production load information, the electricity energy distribution plan, and the corresponding decision-making moment includes: Determine the cost coefficients; wherein, the cost coefficients include the charging cost coefficient per unit time of the energy storage system, the alarm cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, and the maintenance downtime cost coefficient of the energy storage system to be maintained; the cost composition of the energy storage system includes charging cost, alarm cost, discharge reward, and downtime cost; the cost coefficient corresponding to the charging cost is the charging cost coefficient per unit time of the energy storage system, the cost coefficient corresponding to the alarm cost is the alarm cost coefficient per unit time of the energy storage system, the cost coefficient corresponding to the discharge reward is the discharge reward coefficient per unit time of the energy storage system, and the cost coefficient corresponding to the downtime cost is the maintenance downtime cost coefficient of the energy storage system to be maintained; Divide the historical period into multiple historical decision-making cycles according to the decision-making moment; Determine the electricity energy distribution plan corresponding to each of the historical decision-making cycles, and determine the electricity energy distribution type of the electricity energy distribution plan; wherein, the electricity energy distribution type includes the charging process and the discharging process; Determine the cost values within each of the historical decision-making cycles respectively according to the cost coefficients and the electricity energy distribution types of each of the electricity energy distribution plans; Sum up the cost values within each of the historical decision-making cycles to obtain the cumulative cost value within the historical period.

2. The electric energy control method according to claim 1, characterized in that, Obtain the production load information of the user, including: Obtain the historical production load independent variables of the user; wherein, the historical production load independent variables at least include historical temperature, historical production plan, and historical electricity price information; Construct a production load prediction model according to the historical production load independent variables and the linear regression algorithm; Obtain the target production load independent variables of the user within the historical period; Input the target production load independent variables into the production load prediction model to determine the production load information of the user within the historical period.

3. The electric energy control method according to claim 1, characterized in that, Determine the cost values within each of the historical decision-making cycles respectively according to the cost coefficients and the electricity energy distribution types of each of the electricity energy distribution plans, including: When the electricity energy distribution type of the electricity energy distribution plan is the discharging process, determine the corresponding historical decision-making cycle as the discharging decision-making cycle; Determine the discharge weight coefficient, and determine the alarm level, electricity price information, and production load information of the energy storage system within the discharge decision period; Determine the cumulative discharge cost within the discharge decision period according to the discharge weight coefficient, the alarm level of the energy storage system, and the electricity price information within the discharge decision period.

4. The power control method according to claim 3, wherein Determine the cumulative discharge cost within the discharge decision period according to the discharge weight coefficient, the alarm level of the energy storage system, and the electricity price information within the discharge decision period, including: Judge whether the alarm level of the energy storage system within the discharge decision period is greater than the first threshold; If not, determine the cumulative discharge cost within the discharge decision period according to the discharge weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the discharge decision period, the power distribution plan within the discharge decision period, the alarm level of the energy storage system, the electricity price information, and the production load information; If so, determine the first moment of the alarm of the energy storage system within the discharge decision period; Divide the discharge decision period into a first normal sub-period and a first alarm sub-period according to the first moment; Determine the discharge cost within the first normal sub-period according to the discharge weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the first normal sub-period, the power distribution plan within the first normal sub-period, the alarm level of the energy storage system, the electricity price information, and the production load information; Determine the discharge cost within the first alarm sub-period according to the discharge weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the first alarm sub-period, the power distribution plan within the first alarm sub-period, the alarm level of the energy storage system, the electricity price information, and the production load information; Sum up the discharge cost within the first normal sub-period and the discharge cost within the first alarm sub-period to determine the cumulative discharge cost within the discharge decision period.

5. The electric energy control method according to claim 4, characterized in that, Before dividing the discharge decision period into a first normal sub-period and a first alarm sub-period according to the first moment, after determining the first moment of the alarm of the energy storage system within the discharge decision period, it further includes: Judge whether the energy storage system needs to be maintained and shut down after the first moment within the discharge decision period; If not, enter the step of dividing the discharge decision period into a first normal sub-period and a first alarm sub-period according to the first moment; If so, determine the second moment when the energy storage system needs to be maintained and shut down within the discharge decision period; Divide the discharge decision period into a second normal sub-period, a second alarm sub-period, and a maintenance shutdown period according to the first moment and the second moment; Determine the discharge cost within the second normal sub-period according to the discharge weight coefficient, the alarm cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the second normal sub-period, the power distribution plan within the second normal sub-period, the alarm level of the energy storage system, the electricity price information, and the production load information; Determine the discharge cost within the second warning sub - cycle according to the discharge weight coefficient, the warning cost coefficient per unit time of the energy storage system, the discharge reward coefficient per unit time of the energy storage system, the maintenance - waiting shutdown cost coefficient of the energy storage system, the second warning sub - cycle, the power distribution plan within the second warning sub - cycle, the warning level of the energy storage system, the electricity price information, and the production load information; Sum up the discharge cost within the second normal sub - cycle and the discharge cost within the second warning sub - cycle to determine the cumulative discharge cost within the discharge decision cycle.

6. The electric energy control method according to claim 1, wherein Determine the cost values within each historical decision cycle respectively according to the cost coefficient and the power distribution type of each power distribution plan, including: When the power distribution type of the power distribution plan is the charging process, determine the corresponding historical decision cycle as the charging decision cycle; Determine the charging weight coefficient, and determine the warning level of the energy storage system, the electricity price information, and the production load information within the charging decision cycle; Determine the cumulative charging cost within the charging decision cycle according to the charging weight coefficient, the warning level of the energy storage system, and the electricity price information within the charging decision cycle.

7. The power control method according to claim 6, wherein Determine the cumulative charging cost within the charging decision cycle according to the charging weight coefficient, the warning level of the energy storage system, and the electricity price information within the charging decision cycle, including: Judge whether the warning level of the energy storage system within the charging decision cycle is greater than the second threshold; If not, determine the cumulative charging cost within the charging decision cycle according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the charging decision cycle, the power distribution plan within the charging decision cycle, the warning level of the energy storage system, the electricity price information, and the production load information; If so, determine the first moment of the warning of the energy storage system within the charging decision cycle; Divide the charging decision cycle into a first normal sub - cycle and a first warning sub - cycle according to the first moment; Determine the charging cost within the first normal sub - cycle according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the first normal sub - cycle, the power distribution plan within the first normal sub - cycle, the warning level of the energy storage system, the electricity price information, and the production load information; Determine the charging cost within the first warning sub - cycle according to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the first warning sub - cycle, the power distribution plan within the first warning sub - cycle, the warning level of the energy storage system, the electricity price information, and the production load information; Sum up the charging cost within the first normal sub - cycle and the charging cost within the first warning sub - cycle to determine the cumulative charging cost within the charging decision cycle.

8. The electric energy control method according to claim 7, wherein Before dividing the charging decision period into a first normal sub-period and a first warning sub-period according to the first moment, after determining the first moment of energy storage system warning within the charging decision period, it further includes: Judging whether the energy storage system is to be shut down for operation and maintenance after the first moment within the charging decision period; If not, enter the step of dividing the charging decision period into a first normal sub-period and a first warning sub-period according to the first moment; If so, determine the second moment when the energy storage system is to be shut down for operation and maintenance within the charging decision period; According to the first moment and the second moment, divide the charging decision period into a second normal sub-period, a second warning sub-period and an operation and maintenance shutdown period; According to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the second normal sub-period and the power distribution scheme within the second normal sub-period, the energy storage system warning level, the electricity price information and the production load information, determine the charging cost within the second normal sub-period; According to the charging weight coefficient, the charging cost coefficient per unit time of the energy storage system, the warning cost coefficient per unit time of the energy storage system, the operation and maintenance shutdown cost coefficient of the energy storage system, the second warning sub-period and the power distribution scheme within the second warning sub-period, the energy storage system warning level, the electricity price information and the production load information, determine the charging cost within the second warning sub-period; Sum up the charging cost within the second normal sub-period and the charging cost within the second warning sub-period to determine the cumulative charging cost within the charging decision period.

9. The electric energy control method according to any one of claims 1 to 8, characterized in that Determine the target power distribution scheme from each candidate power distribution scheme in the preset policy space according to the cumulative cost value, including: Determine the reinforcement learning optimization target; wherein, the reinforcement learning optimization target is to determine the target power distribution scheme from each candidate power distribution scheme in the preset policy space so as to minimize the expected average cost per unit time of the system in an infinite time interval; Construct a reinforcement learning model according to the reinforcement learning optimization target, the cumulative cost value and each candidate power distribution scheme in the preset policy space; Solve the reinforcement learning model to obtain the target power distribution scheme.

10. The electric energy control method according to claim 9, wherein, Solving the reinforcement learning model includes: Initialize the Q-value table and the average cost, and set the learning step size decay factor, the initial temperature of simulated annealing, the cooling coefficient and the maximum number of iterations; wherein, the Q-value table contains the Q-values of multiple state-action pairs under the average criterion; the state is a joint state composed of electricity price information, energy storage system state information and production load information, and the action is a candidate power distribution scheme; Read the current state at the current decision moment; Determine the current temperature according to the initial temperature of simulated annealing, the cooling coefficient and the current number of iterations, and determine the exploration probability according to the current temperature; Generate a random number and judge whether the random number is less than the exploration probability; If the random number is less than the exploration probability, randomly select an action from the Q-value table; If the random number is not less than the exploration probability, select the action with the minimum cost corresponding to the current state in the Q-value table; Execute the action and determine the cost of the action; Calculate a difference result based on the cost of the action, the average cost, the minimum Q-value of the next state, and the Q-value corresponding to the current state; Update the Q-value corresponding to the current state according to the Q-value corresponding to the current state, the learning step size decay factor, and the difference result; Update the average cost and the learning step size decay factor, and determine whether the current iteration count has reached the maximum iteration count and the Q-value table satisfies the convergence condition; If it is determined that the current iteration count has not reached the maximum iteration count, and / or the Q-value table does not satisfy the convergence condition, then use the next decision moment as the current decision moment, and return to the step of reading the current state at the current decision moment; If it is determined that the current iteration count has reached the maximum iteration count and the Q-value table satisfies the convergence condition, then confirm that the Q-value table converges; Generate a target power distribution plan according to the converged Q-value table.

11. The electric energy control method according to claim 10, wherein Generating a target power distribution plan according to the converged Q-value table includes: Traverse all states in the converged Q-value table; Select the action with the minimum cost in each state based on the converged Q-value table to obtain an action set; Determine the action set as the target power distribution plan.

12. An electric energy control device, characterized in that, Including: An acquisition module for acquiring electricity price information, energy storage system status information, and the production load information of a user within a historical period; A first determination module for determining an electricity power distribution plan and the corresponding decision moment within a historical period according to the electricity price information, the energy storage system status information, and the production load information; wherein, the decision moment represents the moment when the electricity price changes within the historical period and / or the moment when the energy storage system generates an alarm; A second determination module for determining the cumulative cost value within a historical period according to the electricity price information, the energy storage system status information, the production load information, the electricity power distribution plan, and the corresponding decision moment; A third determination module for determining a target electricity power distribution plan from each candidate electricity power distribution plan in a preset policy space according to the cumulative cost value, so as to execute the target electricity power distribution plan in a future period; Wherein, determining the cumulative cost value within a historical period according to the electricity price information, the energy storage system status information, the production load information, the electricity power distribution plan, and the corresponding decision moment includes: Determine the cost coefficients; wherein, the cost coefficients include the charging cost coefficient of the energy storage system per unit time, the warning cost coefficient of the energy storage system per unit time, the discharging reward coefficient of the energy storage system per unit time, and the downtime cost coefficient of the energy storage system to be maintained and operated; the cost composition of the energy storage system includes charging cost, warning cost, discharging reward, and downtime cost; the cost coefficient corresponding to the charging cost is the charging cost coefficient of the energy storage system per unit time, the cost coefficient corresponding to the warning cost is the warning cost coefficient of the energy storage system per unit time, the cost coefficient corresponding to the discharging reward is the discharging reward coefficient of the energy storage system per unit time, and the cost coefficient corresponding to the downtime cost is the downtime cost coefficient of the energy storage system to be maintained and operated; Divide the historical period into multiple historical decision cycles according to the decision-making moment; Determine the power distribution scheme corresponding to each of the historical decision cycles, and determine the power distribution type of the power distribution scheme; wherein, the power distribution type includes the charging process and the discharging process; According to the cost coefficients and the power distribution types of the power distribution schemes, respectively determine the cost values within each of the historical decision cycles; Sum up the cost values within each of the historical decision cycles to obtain the cumulative cost value within the historical period.

13. An electric energy control device, characterized in that, Comprising: A memory for storing a computer program; A processor for implementing the steps of the power control method according to any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor, the steps of the power control method according to any one of claims 1 to 11 are implemented.

Images