Mobile energy storage scheduling method and device based on knowledge constraint depth expectation Q network

Abstract

The invention provides a mobile energy storage scheduling method and device based on a knowledge constraint depth expectation Q network, and studies charging and discharging decisions taking mobile energy storage as a main load support component, including emergency resource space-time coupling, uncertainty of an extreme meteorological scene, energy storage charging and discharging and time-space transfer constraint. Establishing a load recovery-oriented mobile energy storage elastic optimization scheduling model; based on a mobile energy storage elastic optimization scheduling model, integrating emergency resource space-time coupling, uncertainty of an extreme meteorological scene, energy storage charging and discharging and time-space transfer constraints into mathematical derivation of machine learning preprocessing so as to establish an energy storage Markov decision process covering optimization constraint knowledge fusion; and based on an energy storage Markov decision process, establishing an elastic scheduling process training agent based on a depth expectation Q network algorithm so as to optimize a scheduling strategy of the mobile energy storage system. According to the method, the elasticity of the system is enhanced, the calculation efficiency is ensured, and the problem of uncertainty is effectively solved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of power system dispatching, and in particular to a mobile energy storage dispatching method and device based on a knowledge-constrained deep expected Q network. Background Art

[0002] In recent years, extreme weather has occurred frequently, and the operation of power systems with a high proportion of renewable energy often faces extreme scenarios, resulting in load reduction. Therefore, it is imperative to take anti-underload measures for flexible power systems. With the continuous development of modern power systems, the scale and complexity of power grids have increased significantly. As a key component of the power system, the safe and stable operation of regional power grids is directly related to the reliability and economy of the entire power system. However, with the frequent occurrence of extreme weather events caused by climate change and the large-scale access of new energy, the operating environment faced by the power grid has become more complex and unpredictable. In such highly uncertain extreme scenarios, traditional dispatching methods are often difficult to respond effectively, and the flexible dispatching capabilities of the power grid need to be improved urgently.

[0003] Traditional grid dispatching methods mainly rely on deterministic and static optimization models. These models usually assume that the grid operating environment is relatively stable and formulate dispatching strategies based on historical data and experience. However, in the face of sudden extreme events such as natural disasters such as typhoons, blizzards, earthquakes, and the uncertainty of new energy output, the limitations of these methods gradually emerge. Specifically, on the one hand, traditional methods lack the ability to respond quickly to emergencies; on the other hand, they cannot fully consider the dynamics and uncertainty of grid operation, resulting in poor dispatching strategies in practical applications.

[0004] In recent years, with the rapid development of artificial intelligence technology, deep reinforcement learning (DRL) has shown great potential in solving complex dynamic optimization problems. As a typical method of deep reinforcement learning, the Deep Expected Q-Network (DQN) can perform efficient policy learning in high-dimensional state space by combining deep learning and reinforcement learning. DQN can learn and optimize scheduling strategies by continuously interacting with the environment, making it more adaptable and robust in uncertain environments.

[0005] However, there are still many challenges in applying DQN to regional power grid dispatching. On the one hand, the state space of the power grid is complex and high-dimensional, and how to effectively represent and process the operating state of the power grid is a key issue. On the other hand, in extreme scenarios, the operating environment of the power grid changes rapidly, and how DQN responds to emergencies becomes a challenge. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a mobile energy storage scheduling method and device based on a knowledge-constrained deep expected Q network. The method is based on a deep expected Q network, and improves the flexibility and response capability of the power grid by optimizing the charging and discharging decisions of a mobile energy storage system (MES), so as to at least partially solve the above-mentioned problems.

[0007] According to a first aspect of an embodiment of the present invention, a mobile energy storage scheduling method based on a knowledge-constrained deep expected Q network is provided, comprising: establishing a mobile energy storage elastic optimization scheduling model for load recovery based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, the energy storage charging and discharging, and the spatiotemporal transfer constraints; based on the mobile energy storage elastic optimization scheduling model, integrating the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, the energy storage charging and discharging, and the spatiotemporal transfer constraints into the mathematical derivation of machine learning preprocessing to establish an energy storage Markov decision process covering the fusion of optimization constraint knowledge; based on the energy storage Markov decision process, establishing an elastic scheduling process training agent based on a deep expected Q network algorithm to optimize the scheduling strategy of the mobile energy storage system.

[0008] In one implementation, based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, energy storage charging and discharging, and spatiotemporal transfer constraints, a mobile energy storage elastic optimization scheduling model for load recovery is established, which is expressed as:

[0009]

[0010] P t ess =-P t dis +P t char

[0011]

[0012] in, Indicates the power exchanged between the regional power grid and the outside world, represents the charging and discharging power of the MES system at time t, Greater than 0 means charging. Less than 0 means discharge. represents the restored power of the load at time t, Indicates that the power output of new energy is uncertain, s is a random variable related to scenario s, which is used to characterize the uncertainty of new energy processing under extreme weather scenarios. represents the discharge power at time t, represents the charging power at time t, represents the total stored energy of the energy storage system at time t, represents the new energy injected after the MES arrives at time t, which is used to characterize the spatiotemporal coupling characteristics of emergency resources. represents the load power restored by the regional power grid at time t, η char and η dis They represent the charging efficiency and discharging efficiency respectively. The above formulas are used together to characterize the energy storage charging and discharging and time-space transfer constraints.

[0013] In another implementation, the mobile energy storage elastic optimization scheduling model is based on the spatiotemporal coupling of the emergency resources, the uncertainty of the extreme weather scenario, the energy storage charging and discharging, and the spatiotemporal transfer constraints are integrated into the mathematical derivation of machine learning preprocessing to establish an energy storage Markov decision process covering the fusion of optimization constraint knowledge. Based on the energy storage Markov decision process, an elastic scheduling process training agent based on the deep expected Q network algorithm is established to optimize the scheduling strategy of the mobile energy storage system, which is expressed as:

[0014] A ESS =[0,ΔP dis ,2ΔP dis ...,(n-1)ΔP dis ,nΔP dis ,...,P max _ dis ]

[0015] P max_dis =N0ΔP dis

[0016]

[0017] R t =αR t-1 +(1-α)r t

[0018]

[0019] q(s,a;θ′)=r(s,a)+γE(q(s′,a;θ′))

[0020]

[0021] Among them, A ESS represents the ESS power matrix vector, P max _ dis Indicates the maximum discharge power, ΔP dis represents the adjustable discharge range, N0 represents the adjustable discharge gear, r t Represents a single instant reward return, ω i represents the load weight, Indicates the rated power of the load, Γ trepresents the penalty term, R t represents the cumulative return of t iterations, α is the discount rate, q(s,a;θ) represents the Q-value return obtained by executing action a under the certain state s and network parameters θ, E(·) refers to the expected return obtained under the corresponding uncertain state s, and γ is the training discount rate. The above expression is the knowledge equivalent embedding of the mobile energy storage elastic optimization scheduling model, which is specifically expressed as: Formula A ESS =[0,ΔP dis ,2ΔP dis ...,(n-1)ΔP dis ,nΔP dis ,...,P max_dis ] and P max_dis =N0ΔP dis To approximate the discrete decision of mobile energy storage charging and discharging, and R t =αR t-1 +(1-α)r t The first two equations are penalty function corrections for the soft constraints of overcharging and over-discharging of mobile energy storage, and the remaining equations are expected training representations for multi-scenario stochastic optimization. The above equations realize the embedding of knowledge constraints for elastic optimization scheduling in the process of machine learning.

[0022] In another implementation, the objective function of establishing the flexible scheduling process training agent based on the deep expected Q network algorithm is:

[0023]

[0024] Where L(·) is the objective function of stochastic gradient descent, s' and a' represent the state and action at the next moment, respectively, r is the single reward return, a is the action variable, s is the state variable, γ is the training discount rate, Q(s,a) represents the Q value return obtained by executing a in state s, and φ represents the parameters of all network training.

[0025] According to a second aspect of an embodiment of the present invention, a mobile energy storage scheduling device based on a knowledge-constrained deep expected Q network is provided, comprising: a modeling module for establishing a mobile energy storage elastic optimization scheduling model for load recovery based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, the energy storage charging and discharging, and the spatiotemporal transfer constraints; a knowledge fusion module for integrating the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, the energy storage charging and discharging, and the spatiotemporal transfer constraints into the mathematical derivation of machine learning preprocessing based on the mobile energy storage elastic optimization scheduling model, so as to establish an energy storage Markov decision process covering the fusion of optimization constraint knowledge; a training module for establishing an elastic scheduling process training agent based on a deep expected Q network algorithm based on the energy storage Markov decision process, so as to optimize the scheduling strategy of the mobile energy storage system.

[0026] In one implementation, based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, energy storage charging and discharging, and spatiotemporal transfer constraints, a mobile energy storage elastic optimization scheduling model for load recovery is established, which is expressed as:

[0027]

[0028] P t ess =-P t dis +P t char

[0029]

[0030] in, Indicates the power exchanged between the regional power grid and the outside world, represents the charging and discharging power of the MES system at time t, Greater than 0 means charging. Less than 0 means discharge. represents the restored power of the load at time t, Indicates that the power output of new energy is uncertain, s is a random variable related to scenario s, which is used to characterize the uncertainty of new energy processing under extreme weather scenarios. represents the discharge power at time t, represents the charging power at time t, represents the total stored energy of the energy storage system at time t, represents the new energy injected after the MES arrives at time t, which is used to characterize the spatiotemporal coupling characteristics of emergency resources. represents the load power restored by the regional power grid at time t, η char and η dis They represent the charging efficiency and discharging efficiency respectively. The above formulas are used together to characterize the energy storage charging and discharging and time-space transfer constraints.

[0031] In another implementation, the mobile energy storage elastic optimization scheduling model is based on the spatiotemporal coupling of the emergency resources, the uncertainty of the extreme weather scenario, the energy storage charging and discharging, and the spatiotemporal transfer constraints are integrated into the mathematical derivation of machine learning preprocessing to establish an energy storage Markov decision process covering the fusion of optimization constraint knowledge. Based on the energy storage Markov decision process, an elastic scheduling process training agent based on the deep expected Q network algorithm is established to optimize the scheduling strategy of the mobile energy storage system, which is expressed as:

[0032] A ESS =[0,ΔP dis ,2ΔP dis...,(n-1)ΔP dis ,nΔP dis ,...,P max_dis ]

[0033] P max_dis =N0ΔP dis

[0034]

[0035] R t =αR t-1 +(1-α)r t

[0036]

[0037] q(s,a;θ′)=r(s,a)+γE(q(s′,a;θ′))

[0038]

[0039] Among them, A ESS represents the ESS power matrix vector, P max_dis Indicates the maximum discharge power, ΔP dis represents the adjustable discharge range, N0 represents the adjustable discharge gear, r t Represents a single instant reward return, ω i represents the load weight, Indicates the rated power of the load, Γ t represents the penalty term, R t represents the cumulative return of t iterations, α is the discount rate, q(s,a;θ) represents the Q-value return obtained by executing action a under the certain state s and network parameters θ, E(·) refers to the expected return obtained under the corresponding uncertain state s, and γ is the training discount rate. The above expression is the knowledge equivalent embedding of the mobile energy storage elastic optimization scheduling model, which is specifically expressed as: Formula A ESS =[0,ΔP dis ,2ΔP dis ...,(n-1)ΔP dis ,nΔP dis ,...,P max_dis ] and P max_dis =N0ΔP dis To approximate the discrete decision of mobile energy storage charging and discharging, and R t =αR t-1 +(1-α)r t The first two equations are penalty function corrections for the soft constraints of overcharging and over-discharging of mobile energy storage, and the remaining equations are expected training representations for multi-scenario stochastic optimization. The above equations realize the embedding of knowledge constraints for elastic optimization scheduling in the process of machine learning.

[0040] In another implementation, the objective function of establishing the flexible scheduling process training agent based on the deep expected Q network algorithm is:

[0041]

[0042] Where L(·) is the objective function of stochastic gradient descent, s' and a' represent the state and action at the next moment, respectively, r is the single reward return, a is the action variable, s is the state variable, γ is the training discount rate, Q(s,a) represents the Q value return obtained by executing a in state s, and φ represents the parameters of all network training.

[0043] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising a processor and a memory storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes the steps executed by the method of the first aspect.

[0044] According to a fourth aspect of an embodiment of the present invention, there is provided a computer storage medium on which a computer program is stored. When the program is executed by a processor, the method of the first aspect described above is implemented.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] (1) Based on the knowledge constraint fusion method, the present invention introduces random variable factors into the dispatching model, and embeds them into the training and learning process through intelligent agent rule design to balance the load fluctuations in the power system and effectively represent and process the operating status of the power grid (including extreme scenarios).

[0047] (2) The present invention designs a deep expected Q network (DEQN) algorithm to learn and optimize the scheduling strategy from a large amount of uncertain and fluctuating source and load history and prediction data for sudden extreme scenarios, making the scheduling decision more scientific, accurate and robust. Compared with traditional empirical decision-making, this data-driven decision-making method of the present invention can better adapt to the complex and changeable power grid environment and reduce the risk of frequent voltage fluctuations and power outages.

[0048] (3) The solution of the present invention can provide power system operators with more detailed decision support tools, helping operators to make more informed decisions and improve operational efficiency and reliability when facing a changing power market and operating environment. Engineering practitioners can then conduct relevant research work based on this. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0050] Figure 1 It is a flowchart of the steps of the mobile energy storage scheduling method based on the knowledge-constrained deep expected Q network of the present invention;

[0051] Figure 2 The mobile energy storage system rule diagram of the present invention includes (a) the execution logic of the energy storage system and (b) the charge and discharge curve;

[0052] Figure 3 It is a schematic diagram of discrete probabilistic processing of uncertain output of the present invention;

[0053] Figure 4 It is a schematic diagram of the matching result without energy network verification in the calculation example of the present invention;

[0054] Figure 5 It is a schematic diagram of average power of renewable energy and load of the present invention;

[0055] Figure 6 It is a schematic diagram of recovery results in different scenarios of the present invention.

[0056] Figure 7 It is a structural block diagram of a mobile energy storage scheduling device based on a knowledge-constrained deep expected Q network of the present invention;

[0057] Figure 8 The figure is a schematic diagram of the structure of an electronic device of the present invention. DETAILED DESCRIPTION

[0058] In order to have a clearer understanding of the technical features, purposes and effects of the embodiments of the present invention, the specific implementation of the embodiments of the present invention is now described with reference to the accompanying drawings.

[0059] In this document, “exemplary” means “serving as an example, instance or illustration”, and any illustration or implementation described in this document as “exemplary” should not be construed as a more preferred or more advantageous technical solution.

[0060] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0061] The specific implementation of the embodiment of the present invention is further described below in conjunction with the accompanying drawings of the embodiment of the present invention.

[0062] See also Figure 1 The present invention provides a mobile energy storage scheduling method based on a knowledge-constrained deep expected Q network, which mainly includes the following steps:

[0063] Step S1: Based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, energy storage charging and discharging, and spatiotemporal transfer constraints, a mobile energy storage elastic optimization scheduling model for load recovery is established;

[0064] Step S2: Based on the mobile energy storage elastic optimization scheduling model, the spatiotemporal coupling of the emergency resources, the uncertainty of the extreme weather scenario, the energy storage charging and discharging, and the spatiotemporal transfer constraints are integrated into the mathematical derivation of the machine learning preprocessing to establish an energy storage Markov decision process covering the fusion of optimization constraint knowledge;

[0065] Step S3: Based on the energy storage Markov decision process, a flexible scheduling process training agent based on a deep expected Q network algorithm is established to optimize the scheduling strategy of the mobile energy storage system.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] (1) Based on the knowledge constraint fusion method, the present invention introduces random variable factors into the dispatching model, and embeds them into the training and learning process through intelligent agent rule design to balance the load fluctuations in the power system and effectively represent and process the operating status of the power grid (including extreme scenarios).

[0068] (2) The present invention designs a deep expected Q network (DEQN) algorithm to learn and optimize the scheduling strategy from a large amount of uncertain and fluctuating source and load history and prediction data for sudden extreme scenarios, making the scheduling decision more scientific, accurate and robust. Compared with traditional empirical decision-making, this data-driven decision-making method of the present invention can better adapt to the complex and changeable power grid environment and reduce the risk of frequent voltage fluctuations and power outages.

[0069] (3) The solution of the present invention can provide power system operators with more detailed decision support tools, helping operators to make more informed decisions and improve operational efficiency and reliability when facing a changing power market and operating environment. Engineering practitioners can then conduct relevant research work based on this.

[0070] Optionally, based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, energy storage charging and discharging, and spatiotemporal transfer constraints, a mobile energy storage elastic optimization scheduling model for load recovery is established, which is expressed as:

[0071]

[0072] P t ess =-P t dis +P t char

[0073]

[0074] in, Indicates the power exchanged between the regional power grid and the outside world, represents the charging and discharging power of the MES system at time t, Greater than 0 means charging. Less than 0 means discharge. represents the restored power of the load at time t, Indicates that the power output of new energy is uncertain, s is a random variable related to scenario s, which is used to characterize the uncertainty of new energy processing under extreme weather scenarios. represents the discharge power at time t, represents the charging power at time t, represents the total stored energy of the energy storage system at time t, represents the new energy injected after the MES arrives at time t, which is used to characterize the spatiotemporal coupling characteristics of emergency resources. represents the load power restored by the regional power grid at time t, η char and η dis They represent the charging efficiency and discharging efficiency respectively. The above formulas are used together to characterize the energy storage charging and discharging and time-space transfer constraints.

[0075] Optionally, based on the mobile energy storage elastic optimization scheduling model, the spatiotemporal coupling of the emergency resources, the uncertainty of the extreme weather scenario, the energy storage charging and discharging, and the spatiotemporal transfer constraints are integrated into the mathematical derivation of machine learning preprocessing to establish an energy storage Markov decision process covering the fusion of optimization constraint knowledge. Based on the energy storage Markov decision process, an elastic scheduling process training agent based on the deep expected Q network algorithm is established to optimize the scheduling strategy of the mobile energy storage system, which is expressed as:

[0076] A ESS =[0,ΔP dis ,2ΔP dis ...,(n-1)ΔP dis ,nΔP dis ,...,P max_dis ]

[0077] P max_dis =N0ΔP dis

[0078]

[0079] R t =αR t-1 +(1-α)r t

[0080]

[0081] q(s,a;θ′)=r(s,a)+γE(q(s′,a;θ′))

[0082]

[0083] Among them, A ESS represents the ESS power matrix vector, P max _ dis Indicates the maximum discharge power, ΔP dis represents the adjustable discharge range, N0 represents the adjustable discharge gear, r t Represents a single instant reward return, ω i represents the load weight, Indicates the rated power of the load, Γ t represents the penalty term, R t represents the cumulative return of t iterations, α is the discount rate, q(s,a;θ) represents the Q-value return obtained by executing action a under the certain state s and network parameters θ, E(·) refers to the expected return obtained under the corresponding uncertain state s, and γ is the training discount rate. The above expression is the knowledge equivalent embedding of the mobile energy storage elastic optimization scheduling model, which is specifically expressed as: Formula A ESS =[0,ΔP dis ,2ΔPdis ...,(n-1)ΔP dis ,nΔP dis ,...,P max_dis ] and P max_dis =N0ΔP dis To approximate the discrete decision of mobile energy storage charging and discharging, and R t =αR t-1 +(1-α)r t The first two equations are penalty function corrections for the soft constraints of overcharging and over-discharging of mobile energy storage, and the remaining equations are expected training representations for multi-scenario stochastic optimization. The above equations realize the embedding of knowledge constraints for elastic optimization scheduling in the process of machine learning.

[0084] Optionally, the objective function of establishing a flexible scheduling process training agent based on a deep expected Q network algorithm is:

[0085]

[0086] Where L(·) is the objective function of stochastic gradient descent, s' and a' represent the state and action at the next moment, respectively, r is the single reward return, a is the action variable, s is the state variable, γ is the training discount rate, Q(s,a) represents the Q value return obtained by executing a in state s, and φ represents the parameters of all network training.

[0087] Specifically, the scheme of the present invention is further described according to the following examples:

[0088] 1. Optimization modeling of mobile energy storage energy management

[0089] In extreme cases, mobile energy storage plays a pivotal role in supporting the load as a key emergency resource. This paper focuses on using mobile energy storage as the main supporting asset and considers several assumptions in the dispatch process:

[0090] (1) The research focuses on regional power grids without in-depth study of the specific topology or complex power flows of the power grid.

[0091] (2) Ignore the spatial state of scheduling and only emphasize the temporal aspect, where the scheduling time is clearly defined.

[0092] (3) The study assumes that new energy systems such as wind power and photovoltaic power have random characteristics. It does not include the demand-side response process of special loads, such as electric vehicles, commercial and industrial loads.

[0093] The response curve of mobile energy storage in dispatch recovery is represented by the energy transfer curve. Mobile energy storage is connected to the regional power grid through traffic behavior and performs discharge actions, such as Figure 2As shown in (a) of the figure, the energy of the entire energy storage system suddenly surges significantly, indicating the arrival of mobile energy storage. Conversely, the discharge of energy storage corresponds to the gradual decrease of the energy of the entire system, such as Figure 2 As shown in (b) in .

[0094] The regional power system covers energy storage systems, external grid interaction systems, new energy systems and load systems. In extreme cases, mobile energy storage dispatching also achieves energy balance in the regional power system, with minimum load shedding as the priority objective function. The operating cost corresponds to the minimum load shedding loss cost at each level during the dispatch cycle, which can be expressed as:

[0095]

[0096] In the formula, is the rated power of the Class I load at time t, represents the recovery power of the first type load at time t, ω i Represents the unit loss cost of Class I load.

[0097] At the same time, the following regional power system operation constraints must be met to complete the modeling of the mobile energy storage elastic optimization dispatch model for load recovery:

[0098]

[0099] P t ess =-P t dis +P t char

[0100] 0≤P t char ≤β char P t max_char

[0101] 0≤P t dis ≤β dis P t max_dis

[0102] β char +β dis =1

[0103]

[0104] In the formula, Indicates the power exchanged between the regional power grid and the outside world, It represents the charge and discharge power of the MES system at time t. A value greater than 0 indicates charging, and a value less than 0 indicates discharging. represents the restored power of the load at time t, Indicates that the power output of new energy is uncertain, s is a random variable associated with the scenario s, Represents the discharge power at time t. Represents the charging power at time t. Represents the total stored energy of the energy storage system at time t. It represents the new energy injected after the MES arrives at time t.

[0105] 2. Reinforcement Learning Preprocessing for Constrained Knowledge Fusion

[0106] 1. Markov Decision Process

[0107] Think of the interaction between the agent and the environment as a discrete time series. The agent starts with an initial environment s0 and subsequently chooses an action a0. As a result, the environment transitions to a new state s1 and provides the agent with an immediate reward r1. Subsequently, the agent takes action a1a1 based on state s1, causing the environment to transition to s2, accompanied by a feedback reward r2. This repeated interaction continues indefinitely.

[0108] For MES as an agent, a set of actions a1, a2, ..., a n , and a set of local observations O1, O2, …O N . For the action set, the charging and discharging process of the energy storage system is the action behavior of reinforcement learning. For the local observation set, the state of the power system at time t includes the charged state of the energy storage, the real-time load state, the real-time renewable energy output, the load output forecast at the next moment, and the renewable energy output forecast at the next moment. The ESS agent selects actions according to its parameterized strategy to maximize its expected cumulative reward. The agent is then transferred to the next state according to the transition probability.

[0109] The objective function of the optimization model is mapped to the reward function of reinforcement learning, which is used to establish the reinforcement learning Markov decision process. The reward function can be defined as:

[0110]

[0111] Among them, r t represents the reward at time t, Γ t Indicates punishment reward.

[0112] Therefore, the total reward of the scheduling cycle of the entire elastic scheduling process is:

[0113]

[0114] In the Markov decision process, the discount rate is the focus on future rewards, so the real-time reward function is modified as follows:

[0115] R t =αR t-1 +(1-α)r t

[0116] 2. Rule Modeling of Agents and Environments

[0117] When converting an optimization model to a machine learning model, it is crucial to convert the various constraints of the optimization model into actionable conditions suitable for machine learning. This involves considering the conversion method for each constraint individually.

[0118] In actual extreme application scenarios, the entire mobile energy storage system acts as an intelligent agent. In this case, the charging process is ignored and only the discharging process is focused on. Specifically, the discharge power is discretely set to . Therefore, the energy storage discharge action set is:

[0119] A ESS =[0,ΔP dis ,2ΔP dis ...,(n-1)ΔP dis ,nΔP dis ,...,P max _ dis ]

[0120] P max_dis =N0ΔP dis

[0121] 3. Mobile energy storage dispatch solution strategy

[0122] According to the above knowledge constraint fusion method, a network training process for mobile energy storage scheduling is established. During actual output, the regional power grid outputs the mobile or output plan of the energy storage system according to the uncertain output of new energy at the next moment. The specific training process is as follows:

[0123] Given the probability distribution of the assumed additional energy output, the probability is processed according to the discretization method, such as Figure 3 shown.

[0124] In order to calculate the value function in the continuous state and action space, a function Q can be designed φ (s,a)≈Q π (s,a) is used for approximate calculation. The function is a neural network with parameter φ. In order to make the two as equal as possible, the stochastic gradient descent method is used, and the objective function of network training is:

[0125]

[0126] Where L(·) is the objective function of stochastic gradient descent, s' and a' represent the state and action at the next moment, respectively, r is the single reward return, a is the action variable, s is the state variable, γ is the training discount rate, Q(s,a) represents the Q value return obtained by executing a in state s, and φ represents the parameters of all network training.

[0127] Deep Q Network uses two strategies:

[0128] The first is to freeze the target network, that is, to keep the target parameters fixed for a certain period of time to stabilize the learning objectives;

[0129] The second is experience replay, which eliminates data correlation by establishing an experience pool (replay buffer). This experience pool contains the experience that the agent has recently encountered.

[0130] First, the dispatch strategy of the mobile energy storage system is selected in the estimated Q network. Then, the value function is updated in the target Q network, as shown below. The probability density function is discretized.

[0131]

[0132] q(s,a;θ′)=r(s,a)+γE(q(s′,a;θ′))

[0133]

[0134] The specific algorithm flow is shown in Table 1:

[0135] Table 1

[0136]

[0137] (IV) Example Analysis

[0138] The scheduling is based on a 24-hour time scale, and the hourly output follows a certain probability distribution. Using the Monte Carlo method, 1,000 sets of possible output data are randomly generated every hour. The fluctuation of 10% above and below is used as the 95% confidence interval of the normal distribution and is set as a random value. The average power of renewable energy and load is as follows Figure 4 shown.

[0139] The present invention uses Python language and calls PyTorch package to write the model. The computer CPU model is i7-8550U and the memory is 8GB.

[0140] In order to verify the effectiveness of the model and algorithm proposed in the present invention, the present invention designs the following four modes for comparative analysis:

[0141] Mode 1: Considering the uncertainty of load output, simulation is performed according to the algorithm proposed by the model.

[0142] Mode 2: Use traditional optimization methods for simulation without considering the uncertainty of load output.

[0143] Mode 3: Similar to Mode 1, this mode also takes into account the uncertainty of load output. However, it uses traditional optimization methods for simulation.

[0144] Figure 5 The results of energy storage dispatch under different scenarios are given, and the figure describes the process of energy storage discharge power gradually decreasing at first and then suddenly increasing. This pattern is consistent with the relatively low renewable energy output at night and the sudden increase in load power during peak demand, which is consistent with expectations.

[0145] Compared with Mode 2 and Mode 3, the mode proposed by the present invention has lower discharge power at the beginning and higher discharge power at the end. This difference is due to the larger power gap observed later, allowing Mode 1 to retain a larger uncertainty.

[0146] Figure 6 The simulation results of recovery load under different scenarios are shown in Figure 2. In the case of average scenario optimization, for example Figure 6 As shown in (a), mode 2 has better fault recovery, but the overall difference is not significant. In extreme random fields, the overall performance of modes 1 and 3 is better than that of mode 2.

[0147] Specifically, in five randomly selected scenarios, e.g. Figure 6 As shown in (b), (c), (d), and (e), the load levels of Mode 1 and Mode 3 are higher than those of Mode 2. Mode 1 and Mode 3 each have their own advantages and disadvantages. This is because Model 2 does not consider uncertainty. Both Model 1 and Model 3 consider uncertainty, but the solution methods are different.

[0148] See Table 2 for the calculation time and objective function value of each mode. Compared with Mode 1 and Mode 2, Mode 3 takes longer to calculate, which indicates that the traditional stochastic optimization model is not suitable for solving stochastic optimization problems with a large number of uncertain scenarios. Due to the consideration of uncertainty, the objective function value of Mode 1 is lower than that of Mode 2, and a relatively conservative result is obtained at the expense of a part of the objective function value.

[0149] Table 2

[0150] model 1 2 3 Solution time / second 36.8s 14.2s >1000s Objective Function 131.4 112.1 /

[0151] In summary, the mobile energy storage scheduling method based on knowledge-constrained deep expected Q network proposed in the present invention studies the charging and discharging decisions with mobile energy storage as the main load supporting component. First, a mobile energy storage elastic optimization scheduling model for load recovery is established, highlighting the impact of the spatiotemporal transfer characteristics of mobile energy storage (MES) on the entire system. Secondly, a Markov decision process of the mobile energy storage system as an intelligent agent is established, and the optimization model constraints are integrated into the mathematical derivation of machine learning preprocessing. Finally, a flexible scheduling process based on a deep Q network algorithm is proposed. Simulation results show that the algorithm and model of the present invention enhance the elasticity of the system, ensure computational efficiency, and effectively solve the uncertainty problem.

[0152] As another example, see Figure 7 The present invention provides a mobile energy storage scheduling device 700 based on a knowledge-constrained deep expected Q network, comprising:

[0153] Modeling module 710, for establishing a mobile energy storage elastic optimization scheduling model for load recovery based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, energy storage charging and discharging, and spatiotemporal transfer constraints;

[0154] The knowledge fusion module 720 is used to integrate the spatiotemporal coupling of the emergency resources, the uncertainty of the extreme weather scenario, the energy storage charging and discharging, and the spatiotemporal transfer constraints into the mathematical derivation of the machine learning preprocessing based on the mobile energy storage elastic optimization scheduling model, so as to establish an energy storage Markov decision process covering the optimization constraint knowledge fusion;

[0155] The training module 730 is used to establish an elastic scheduling process training agent based on the deep expected Q network algorithm based on the energy storage Markov decision process to optimize the scheduling strategy of the mobile energy storage system.

[0156] Optionally, based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, energy storage charging and discharging, and spatiotemporal transfer constraints, a mobile energy storage elastic optimization scheduling model for load recovery is established, which is expressed as:

[0157]

[0158] P t ess =-P t dis +P t char

[0159]

[0160] in, Indicates the power exchanged between the regional power grid and the outside world, represents the charging and discharging power of the MES system at time t, Greater than 0 means charging. Less than 0 means discharge. represents the restored power of the load at time t, Indicates that the power output of new energy is uncertain, s is a random variable related to scenario s, which is used to characterize the uncertainty of new energy processing under extreme weather scenarios. represents the discharge power at time t, represents the charging power at time t, represents the total stored energy of the energy storage system at time t, represents the new energy injected after the MES arrives at time t, which is used to characterize the spatiotemporal coupling characteristics of emergency resources. represents the load power restored by the regional power grid at time t, η char and η dis They represent the charging efficiency and discharging efficiency respectively. The above formulas are used together to characterize the energy storage charging and discharging and time-space transfer constraints.

[0161] Optionally, based on the mobile energy storage elastic optimization scheduling model, the spatiotemporal coupling of the emergency resources, the uncertainty of the extreme weather scenario, the energy storage charging and discharging, and the spatiotemporal transfer constraints are integrated into the mathematical derivation of machine learning preprocessing to establish an energy storage Markov decision process covering the fusion of optimization constraint knowledge. Based on the energy storage Markov decision process, an elastic scheduling process training agent based on the deep expected Q network algorithm is established to optimize the scheduling strategy of the mobile energy storage system, which is expressed as:

[0162] A ESS =[0,ΔP dis ,2ΔP dis ...,(n-1)ΔP dis ,nΔP dis ,...,P max _ dis ]

[0163] P max_dis =N0ΔP dis

[0164]

[0165] R t =αR t-1 +(1-α)r t

[0166]

[0167] q(s,a;θ′)=r(s,a)+γE(q(s′,a;θ′))

[0168]

[0169] Among them, A ESS represents the ESS power matrix vector, P max_dis Indicates the maximum discharge power, ΔP dis represents the adjustable discharge range, N0 represents the adjustable discharge gear, r t represents a single immediate reward return, ψ i represents the load weight, Indicates the rated power of the load, Γ t represents the penalty term, R t represents the cumulative return of t iterations, α is the discount rate, q(s,a;θ) represents the Q-value return obtained by executing action a under the certain state s and network parameters θ, E(·) refers to the expected return obtained under the corresponding uncertain state s, and γ is the training discount rate. The above expression is the knowledge equivalent embedding of the mobile energy storage elastic optimization scheduling model, which is specifically expressed as: Formula A ESS =[0,ΔP dis ,2ΔP dis ...,(n-1)ΔP dis ,nΔP dis ,...,P max_dis ] and P max_dis =N0ΔP dis To approximate the discrete decision of mobile energy storage charging and discharging, and R t =αR t-1 +(1-α)r t The first two equations are penalty function corrections for the soft constraints of overcharging and over-discharging of mobile energy storage, and the remaining equations are expected training representations for multi-scenario stochastic optimization. The above equations realize the embedding of knowledge constraints for elastic optimization scheduling in the process of machine learning.

[0170] Optionally, the objective function of establishing a flexible scheduling process training agent based on a deep expected Q network algorithm is:

[0171]

[0172] Where L(·) is the objective function of stochastic gradient descent, s' and a' represent the state and action at the next moment, respectively, r is the single reward return, a is the action variable, s is the state variable, γ is the training discount rate, Q(s,a) represents the Q value return obtained by executing a in state s, and φ represents the parameters of all network training.

[0173] It should be understood that the mobile energy storage scheduling device based on the knowledge-constrained deep expected Q network of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments.

[0174] As another example, see Figure 8 , an electronic device 800 is provided, and a structural block diagram of the electronic device 800 that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0175] The electronic device 800 may include: a processor (processor) 802 , a communication interface (Communications Interface) 804 , a memory (memory) 806 , and a communication bus 808 .

[0176] The processor 802, the communication interface 804, and the memory 806 communicate with each other via a communication bus 808. The communication interface 804 is used to communicate with other electronic devices or servers.

[0177] The processor 802 is used to execute the program 810, and specifically can execute the relevant steps in the above method embodiment.

[0178] Specifically, the program 810 may include program codes, which include computer operation instructions.

[0179] The processor 802 may be a processor CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0180] The memory 806 is used to store the program 810. The memory 806 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0181] When executed by processor 802, program 810 is used to enable the electronic device to execute a mobile energy storage scheduling method based on a knowledge-constrained deep expected Q network, including establishing a mobile energy storage elastic optimization scheduling model for load recovery based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, energy storage charging and discharging, and spatiotemporal transfer constraints; based on the mobile energy storage elastic optimization scheduling model, the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, the energy storage charging and discharging, and the spatiotemporal transfer constraints are integrated into the mathematical derivation of machine learning preprocessing to establish an energy storage Markov decision process covering the fusion of optimization constraint knowledge; based on the energy storage Markov decision process, an elastic scheduling process training agent based on a deep expected Q network algorithm is established to optimize the scheduling strategy of the mobile energy storage system.

[0182] In addition, the specific implementation of each step in program 810 can refer to the corresponding description of the corresponding steps and units in the above method embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0183] The exemplary embodiments of the present invention further provide a computer storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the methods of the various embodiments of the present invention. The corresponding process descriptions in the aforementioned method embodiments may be referred to and will not be repeated here.

[0184] The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code that may be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded over a network and will be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, processor, or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown herein.

[0185] Thus far, specific embodiments of the present invention have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired results. Additionally, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing may be advantageous.

[0186] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0187] Finally, it should be noted that the above implementation methods are only used to illustrate the embodiments of the present invention, and are not limitations of the embodiments of the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present invention. The patent protection scope of the embodiments of the present invention should be defined by the claims.

Claims

1. A mobile energy storage scheduling method based on knowledge-constrained deep expected Q network, characterized in that: include: Based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, energy storage charging and discharging, and spatiotemporal transfer constraints, a mobile energy storage elastic optimization scheduling model for load recovery is established; Based on the mobile energy storage elastic optimization scheduling model, the spatiotemporal coupling of the emergency resources, the uncertainty of the extreme weather scenario, the energy storage charging and discharging, and the spatiotemporal transfer constraints are integrated into the mathematical derivation of machine learning preprocessing to establish an energy storage Markov decision process that covers the fusion of optimization constraint knowledge; Based on the energy storage Markov decision process, an elastic scheduling process training agent based on the deep expected Q network algorithm is established to optimize the scheduling strategy of the mobile energy storage system.

2. The method according to claim 1, characterized in that Based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, energy storage charging and discharging, and spatiotemporal transfer constraints, a mobile energy storage elastic optimization scheduling model for load recovery is established, which is expressed as: P t ess =-P t dis +P t char in, Indicates the power exchanged between the regional power grid and the outside world, represents the charging and discharging power of the MES system at time t, Greater than 0 means charging. Less than 0 means discharge. represents the restored power of the load at time t, Indicates that the power output of new energy is uncertain, s is a random variable related to scenario s, which is used to characterize the uncertainty of new energy processing under extreme weather scenarios. represents the discharge power at time t, represents the charging power at time t, represents the total stored energy of the energy storage system at time t, represents the new energy injected after the MES arrives at time t, which is used to characterize the spatiotemporal coupling characteristics of emergency resources. represents the load power restored by the regional power grid at time t, η char and η dis They represent the charging efficiency and discharging efficiency respectively. The above formulas are used together to characterize the energy storage charging and discharging and time-space transfer constraints.

3. The method according to claim 2, characterized in that The elastic optimization scheduling model based on the mobile energy storage integrates the spatiotemporal coupling of the emergency resources, the uncertainty of the extreme weather scenario, the energy storage charging and discharging, and the spatiotemporal transfer constraints into the mathematical derivation of machine learning preprocessing to establish an energy storage Markov decision process covering the fusion of optimization constraint knowledge. Based on the energy storage Markov decision process, an elastic scheduling process training agent based on the deep expected Q network algorithm is established to optimize the scheduling strategy of the mobile energy storage system, which is expressed as: A ESS =[0,ΔP dis ,2ΔP dis ...,(n-1)ΔP dis ,nΔP dis ,...,P max_dis ] P max_dis =N0ΔP dis R t =αR t-1 +(1-a)r t q(s,a;θ′)=r(s,a)+γE(q(s′,a;θ′)) Among them, A ESS represents the ESS power matrix vector, P max_dis Indicates the maximum discharge power, ΔP dis represents the adjustable discharge range, N0 represents the adjustable discharge gear, r t Represents a single instant reward return, ω i represents the load weight, Indicates the rated power of the load, Γ t represents the penalty term, R t represents the cumulative return of t iterations, α is the discount rate, q(s,a;θ) represents the Q-value return obtained by executing action a under the certain state s and network parameters θ, E(·) refers to the expected return obtained under the corresponding uncertain state s, and γ is the training discount rate. The above expression is the knowledge equivalent embedding of the mobile energy storage elastic optimization scheduling model, which is specifically expressed as: Formula A ESS =[0,ΔP dis ,2ΔP dis ...,(n-1)ΔP dis ,nΔP dis ,...,P max_dis ] and P max_dis =N0ΔP dis To approximate the discrete decision of mobile energy storage charging and discharging, and R t =αR t-1 +(1-α)r t The first two equations are penalty function corrections for the soft constraints of overcharging and over-discharging of mobile energy storage, and the remaining equations are expected training representations for multi-scenario stochastic optimization. The above equations realize the embedding of knowledge constraints for elastic optimization scheduling in the process of machine learning.

4. The method according to claim 1, characterized in that: The objective function of establishing the flexible scheduling process training agent based on the deep expected Q network algorithm is: Where L(·) is the objective function of stochastic gradient descent, s' and a' represent the state and action at the next moment, respectively, r is the single reward return, a is the action variable, s is the state variable, γ is the training discount rate, Q(s,a) represents the Q value return obtained by executing a in state s, and φ represents the parameters of all network training.

5. A mobile energy storage dispatching device based on knowledge-constrained deep expectation Q network, characterized in that: include: Modeling module, which is used to establish a mobile energy storage elastic optimization scheduling model for load recovery based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, and the constraints of energy storage charging and discharging and spatiotemporal transfer; A knowledge fusion module is used to integrate the spatiotemporal coupling of the emergency resources, the uncertainty of the extreme weather scenario, the energy storage charging and discharging, and the spatiotemporal transfer constraints into the mathematical derivation of machine learning preprocessing based on the mobile energy storage elastic optimization scheduling model, so as to establish an energy storage Markov decision process covering the optimization constraint knowledge fusion; A training module is used to establish an elastic scheduling process training agent based on a deep expected Q network algorithm based on the energy storage Markov decision process to optimize the scheduling strategy of the mobile energy storage system.

6. The device according to claim 5, characterized in that Based on the spatiotemporal coupling of emergency resources, the uncertainty of extreme weather scenarios, energy storage charging and discharging, and spatiotemporal transfer constraints, a mobile energy storage elastic optimization scheduling model for load recovery is established, which is expressed as: P t ess =-P t dis +P t char in, Indicates the power exchanged between the regional power grid and the outside world, represents the charging and discharging power of the MES system at time t, Greater than 0 means charging. Less than 0 means discharge. represents the restored power of the load at time t, Indicates that the power output of new energy is uncertain, s is a random variable related to scenario s, which is used to characterize the uncertainty of new energy processing under extreme weather scenarios. represents the discharge power at time t, represents the charging power at time t, represents the total stored energy of the energy storage system at time t, represents the new energy injected after the MES arrives at time t, which is used to characterize the spatiotemporal coupling characteristics of emergency resources. represents the load power restored by the regional power grid at time t, η char and η dis They represent the charging efficiency and discharging efficiency respectively. The above formulas are used together to characterize the energy storage charging and discharging and time-space transfer constraints.

7. The device according to claim 6, characterized in that The elastic optimization scheduling model based on the mobile energy storage integrates the spatiotemporal coupling of the emergency resources, the uncertainty of the extreme weather scenario, the energy storage charging and discharging, and the spatiotemporal transfer constraints into the mathematical derivation of machine learning preprocessing to establish an energy storage Markov decision process covering the fusion of optimization constraint knowledge. Based on the energy storage Markov decision process, an elastic scheduling process training agent based on the deep expected Q network algorithm is established to optimize the scheduling strategy of the mobile energy storage system, which is expressed as: A ESS =[0,ΔP dis ,2ΔP dis ...,(n-1)ΔP dis ,nΔP dis ,...,P max _ dis ] P max_dis =N0ΔP dis R t =αR t-1 +(1-a)r t q(s,a;θ′)=r(s,a)+γE(q(s′,a;θ′)) Among them, A ESS represents the ESS power matrix vector, P max_dis Indicates the maximum discharge power, ΔP dis represents the adjustable discharge range, N0 represents the adjustable discharge gear, r t Represents a single instant reward return, ω i represents the load weight, Indicates the rated power of the load, Γ t represents the penalty term, R t represents the cumulative reward of t iterations, Δ is the discount rate, q(s,a;θ) represents the Q-value reward obtained by executing action a under the certain state s and network parameters θ, E(·) refers to the expected reward obtained under the corresponding uncertain state s, and γ is the training discount rate. The above expression is the knowledge equivalent embedding of the mobile energy storage elastic optimization scheduling model, which is specifically expressed as: Formula A ESS =[0,ΔP dis , 2ΔP dis ..., (n-1)ΔP dis , nΔP dis , ..., P max_dis ] and P max_dis =N0ΔP dis To approximate the discrete decision of mobile energy storage charging and discharging, and R t =αR t-1 +(1-α)r t The first two equations are penalty function corrections for the soft constraints of overcharging and over-discharging of mobile energy storage, and the remaining equations are expected training representations for multi-scenario stochastic optimization. The above equations realize the embedding of knowledge constraints for elastic optimization scheduling in the process of machine learning.

8. The device according to claim 5, characterized in that The objective function of establishing the flexible scheduling process training agent based on the deep expected Q network algorithm is: Where L(·) is the objective function of stochastic gradient descent, s' and a' represent the state and action at the next moment, respectively, r is the single reward return, a is the action variable, s is the state variable, γ is the training discount rate, Q(s,a) represents the Q value return obtained by executing a in state s, and φ represents the parameters of all network training.

9. An electronic device, characterized in that: include: processor; A memory for storing programs; The program includes instructions, which, when executed by the processor, cause the processor to perform the steps of the method as claimed in any one of claims 1 to 4.

10. A computer storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images