Community-level electric vehicle-household energy storage cooperative scheduling method and system

Abstract

The invention relates to the technical field of power distribution and utilization, and provides a community-level electric vehicle-household energy storage collaborative scheduling method and system. The method comprises the following steps: dividing real-time electricity price, battery SOC and external temperature into a plurality of fuzzy subsets, and determining a membership function of each fuzzy subset based on a TSK fuzzy mathematical model; introducing a price sensitivity parameter, a battery state parameter and a temperature sensitivity parameter, formulating a fuzzy rule of each fuzzy subset in three dimensions, and quantitatively evaluating the response willingness of the electric vehicle through fuzzy reasoning of all membership functions; an SAC algorithm is adopted, a state space and an action space are respectively designed according to the community environment, and an overall reward function is constructed by combining power grid benefit rewards, user comfort rewards and electric vehicle demand response rewards; and inputting a quantitative evaluation result into an SAC algorithm to obtain an optimal scheduling strategy of the energy storage resources. According to the invention, collaborative scheduling optimization of the electric vehicle cluster and the household energy storage device is realized.

Description

Technical Field

[0001] The present invention relates to the field of power distribution technology, and in particular to a community-level electric vehicle-household energy storage coordinated scheduling method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Community energy management plays a key role in integrating renewable energy. In the community energy management system, the coordinated configuration of renewable energy and energy storage devices enhances the flexibility of system scheduling, but also brings uncertainty and complex constraints. With the increase in electric vehicles, the capacity of power batteries is also increasing. The capacity of power batteries is the most potential flexibility resource for new power systems, but the current proportion of vehicle-grid interaction is extremely low. In this context, building a community energy management system that takes into account the participation of electric vehicle clusters and realizing the coordinated optimization of distributed energy and electric vehicles is a technical difficulty that urgently needs to be broken through in the field of power distribution and use. The design of the community energy management system needs to comprehensively consider multiple factors such as the response willingness of electric vehicles, the characteristics of energy storage systems, optimization algorithms, and system operation. It is a key technology for achieving zero-carbon communities and improving energy efficiency.

[0004] Community energy management involves the coordinated scheduling between multiple types of users and devices, and its complexity is reflected in many aspects. First, as an important flexible load resource, the response ability of electric vehicles is significantly affected by the subjective willingness of users, and an accurate response willingness evaluation model needs to be established. At present, electric vehicle response evaluation is mainly divided into two categories: economic model-based and psychological model-based. The economic model analyzes the impact of price changes on user willingness through utility functions, while the psychological model analyzes user behavior preferences based on questionnaires. Secondly, community energy management faces serious uncertainty problems, including the volatility of photovoltaic power generation and the randomness of user behavior. To meet this challenge, researchers have proposed scheduling models based on robust optimization, probabilistic prediction and other methods. In addition, with the application of reinforcement learning theory in the energy field, more and more studies have begun to use deep reinforcement learning methods to achieve community energy optimization scheduling.

[0005] At present, the technical solution closest to the present invention is the rule-based community energy management method. This method realizes system scheduling through preset thresholds and simple control strategies, but has the following limitations: the rule-based control method has poor adaptability to complex environments and is difficult to handle multi-objective and multi-constrained optimization problems; the user response willingness evaluation method is too simplified and does not fully consider multi-dimensional influencing factors such as temperature and battery status; the energy management system rarely considers the participation of electric vehicles, resulting in insufficient utilization of community regulation resources. Summary of the invention

[0006] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a community-level electric vehicle-household energy storage coordinated scheduling method and system. The present invention realizes the coordinated scheduling optimization of electric vehicle clusters and household energy storage devices through TSK fuzzy mathematical evaluation and soft actor-critic (SAC) reinforcement learning algorithm, thereby improving the energy utilization efficiency and economic benefits of the community.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A first aspect of the present invention provides a community-level electric vehicle-household energy storage coordinated scheduling method.

[0009] A community-level electric vehicle-household energy storage coordinated dispatching method, comprising:

[0010] The real-time electricity price, battery SOC and external temperature are divided into several fuzzy subsets. Based on the TSK fuzzy mathematical model, the membership function of each fuzzy subset is determined. The price sensitivity parameter, battery state parameter and temperature sensitivity parameter are introduced to formulate fuzzy rules for each fuzzy subset in three dimensions. Through fuzzy reasoning of all membership functions, the response willingness of electric vehicles is quantitatively evaluated.

[0011] The SAC algorithm is used to design the state space and action space according to the community environment. The overall reward function is constructed by combining grid benefit rewards, user comfort rewards and electric vehicle demand response rewards. The quantitative evaluation results are input into the SAC algorithm to obtain the optimal scheduling strategy for energy storage resources.

[0012] Furthermore, the real-time electricity price, battery SOC and outside temperature are divided into several fuzzy subsets, and the method includes: dividing the real-time electricity price into three fuzzy subsets of low, medium and high, dividing the battery SOC into three fuzzy subsets of insufficient, medium and sufficient, and dividing the outside temperature into three fuzzy subsets of cold, comfortable and hot.

[0013] Furthermore, the method of determining the membership function of each fuzzy subset based on the TSK fuzzy mathematical model includes: determining four key parameters of the membership function based on the TSK fuzzy mathematical model, and establishing the membership function of each fuzzy subset.

[0014] Furthermore, the method of quantitatively evaluating the response willingness of electric vehicles by fuzzy reasoning of all membership functions includes: calculating the weighted average of all fuzzy rule output results by fuzzy reasoning of all membership functions, and calculating the response willingness of electric vehicles.

[0015] Furthermore, the state space includes: environmental state and building state. The environmental state includes: date type, time, direct solar irradiance, real-time carbon emission coefficient and real-time electricity price. The building state includes: non-transferable load, photovoltaic power generation, energy storage battery charge state, net power consumption and electric vehicle battery charge state.

[0016] Furthermore, the action space and method include: charging and discharging control decisions of building fixed energy storage devices and electric vehicle batteries.

[0017] Furthermore, the overall reward function is expressed by the following formula:

[0018] R t =R g +ω c R c +ω v R EV

[0019] In the formula, R t represents the total reward function, R g represents the grid benefit reward function, R c represents the user comfort reward function, R EV represents the electric vehicle reward function, ω c and ω v are the weight coefficients of user comfort and electric vehicle items respectively.

[0020] Furthermore, during the training process of the SAC algorithm, the Actor network of the SAC algorithm is evaluated, and the continuous actions of the energy storage control strategy in the community are generated based on the current state. The value of the action is evaluated by the Critic network, and the target network gradually tracks the parameters of the SAC algorithm through the soft update mechanism.

[0021] Furthermore, during the training process of the SAC algorithm, evaluation indicators are designed to evaluate the performance of the SAC algorithm, and the evaluation indicators include: power consumption, power cost, carbon emission, net power consumption, daily average peak, power ramp and 1-load factor.

[0022] Furthermore, during the training process of the SAC algorithm, the performance of the SAC algorithm is optimized by adjusting the network parameters or the reward function weights of the SAC algorithm.

[0023] Furthermore, during the training process of the SAC algorithm, hyperparameters and reference baselines are designed for SAC algorithm reinforcement learning and performance evaluation.

[0024] A second aspect of the present invention provides a community-level electric vehicle-household energy storage coordinated scheduling system.

[0025] A community-level electric vehicle-household energy storage coordinated dispatching system, comprising:

[0026] The fuzzy processing module is configured to: divide the real-time electricity price, battery SOC and external temperature into several fuzzy subsets, determine the membership function of each fuzzy subset based on the TSK fuzzy mathematical model; introduce price sensitivity parameters, battery state parameters and temperature sensitivity parameters, formulate fuzzy rules for each fuzzy subset in three dimensions, and quantitatively evaluate the response willingness of electric vehicles through fuzzy reasoning of all membership functions;

[0027] The reinforcement learning module is configured as follows: using the SAC algorithm, designing the state space and action space respectively according to the community environment, and constructing an overall reward function combining grid benefit rewards, user comfort rewards and electric vehicle demand response rewards; inputting the quantitative evaluation results into the SAC algorithm to obtain the optimal scheduling strategy for energy storage resources.

[0028] A third aspect of the present invention provides a computer-readable storage medium.

[0029] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the community-level electric vehicle-household energy storage coordinated scheduling method as described in the first aspect above.

[0030] A fourth aspect of the present invention provides a computer device.

[0031] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the community-level electric vehicle-household energy storage coordinated scheduling method as described in the first aspect above are implemented.

[0032] A fifth aspect of the present invention provides a computer program product or a computer program.

[0033] The present invention provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the community-level electric vehicle-household energy storage coordinated scheduling method as described in the first aspect above.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The present invention proposes a community-level electric vehicle-household energy storage collaborative scheduling method and system, which accurately quantifies the response willingness of electric vehicles through the TSK fuzzy mathematical model, and uses the SAC reinforcement learning algorithm to achieve collaborative optimization of energy storage and electric vehicles. It can not only accurately evaluate the response capability of electric vehicles, but also significantly improve the efficiency of community energy management, while ensuring user comfort and system economy. On the basis of maintaining the simplicity of the rule-based control method, the present invention effectively solves the problems of environmental adaptability, response willingness quantification and collaborative optimization by introducing fuzzy mathematics and reinforcement learning methods, providing a more intelligent, efficient and economical solution for community energy management. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0037] Figure 1 It is a flow chart of a community-level electric vehicle-household energy storage coordinated dispatching method shown in the present invention;

[0038] Figure 2 is a flow chart of quantifying the response willingness of electric vehicles shown in the present invention;

[0039] Figure 3 is a schematic diagram of the real-time electricity price membership function shown in the present invention;

[0040] Figure 4 is a schematic diagram of the quantification result of the response willingness of electric vehicles shown in the present invention;

[0041] Figure 5 It is a schematic diagram of the SAC algorithm architecture shown in the present invention;

[0042] Figure 6 Schematic diagram of the relationship between the reward function value and the number of training rounds shown in the present invention;

[0043] Figure 7 It is a schematic diagram comparing the SAC strategy and the RBC strategy shown in the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0045] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0047] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0048] Embodiment 1

[0049] like Figure 1 As shown, this embodiment provides a community-level electric vehicle-household energy storage collaborative scheduling method. This embodiment uses the method applied to a server as an example for illustration. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0050] The real-time electricity price, battery SOC and external temperature are divided into several fuzzy subsets. Based on the TSK fuzzy mathematical model, the membership function of each fuzzy subset is determined. The price sensitivity parameter, battery state parameter and temperature sensitivity parameter are introduced to formulate fuzzy rules for each fuzzy subset in three dimensions. Through fuzzy reasoning of all membership functions, the response willingness of electric vehicles is quantitatively evaluated.

[0051] The SAC algorithm is used to design the state space and action space according to the community environment. The overall reward function is constructed by combining grid benefit rewards, user comfort rewards and electric vehicle demand response rewards. The quantitative evaluation results are input into the SAC algorithm to obtain the optimal scheduling strategy for energy storage resources.

[0052] In view of the limitations of traditional rule-based control methods in terms of adaptability to complex environments, quantification of response willingness, and collaborative optimization, as well as the problem that existing methods are difficult to fully tap the regulation potential of electric vehicles, the present invention provides a community-level electric vehicle-household energy storage collaborative scheduling method and system, which accurately quantifies the response willingness of electric vehicle users through the TSK fuzzy model, uses the SAC reinforcement learning algorithm to achieve collaborative optimization of energy storage and electric vehicles, and improves the system's adaptive ability by building an Actor-Critic dual network architecture. The present invention improves the observability of electric vehicle response capabilities and achieves efficient energy utilization while ensuring user comfort and system economy. The present invention can improve the intelligence level and practicality of the community energy management model, effectively coordinate community benefits and user experience, and ultimately promote energy conservation and emission reduction in the community.

[0053] The present embodiment is described in detail below with reference to the accompanying drawings. Figure 1 As shown:

[0054] Step 1: Quantification of electric vehicle user response willingness. According to the influence of real-time electricity price, electric vehicle battery SOC and external temperature on user response willingness, a TSK fuzzy mathematical model is established to quantitatively evaluate user response willingness.

[0055] With the rapid development of electric vehicles, they are becoming an increasingly important flexible regulation resource. The accurate quantification of their response capability is the basis for accurate load forecasting and optimized scheduling. The response capability of electric vehicles is affected by the owner's willingness to respond. The user's willingness to respond is closely related to people's complex behavior and psychological activities, and has a certain degree of uncertainty. It is difficult to accurately quantify it using a definite traditional mathematical model. The TSK fuzzy mathematical model can be used to solve this problem. The specific process is as follows: Figure 2 shown.

[0056] Step 1.1: Data input: Input real-time electricity price, electric vehicle battery SOC, and external temperature data in hours.

[0057] The response willingness of electric vehicles is affected by real-time electricity prices, battery SOC, and outside temperature. When the electricity price is high, users are more inclined to sell it, and when the electricity price is low, users are more inclined to charge themselves; when the temperature is comfortable, the user's response willingness is the highest. When the outside is cold or hot, electric vehicle users will have the need for electric auxiliary heating or cooling, which will increase the user's mileage anxiety; when the electric vehicle battery SOC is higher, the user's charging demand is reduced, and the response willingness is higher.

[0058] Step 1.2: Divide the fuzzy subsets. The three types of data collected in step 1.1 are divided into three fuzzy subsets respectively. The real-time electricity price is divided into low, medium, and high; the battery SOC is divided into insufficient, medium, and sufficient; the outside temperature is divided into cold, comfortable, and hot.

[0059] Table 1 Fuzzy subset partitioning results (a) Fuzzy subset of electric vehicle charging price

[0060]

[0061] (b) Fuzzy subset of electric vehicle SoC

[0062]

[0063] (c) Fuzzy subset of the outside temperature

[0064]

[0065]

[0066] The fuzzy subset partitioning results are shown in Table 1.

[0067] Step 1.3: Determination of the membership function. According to the fuzzy subsets divided in step 1.2, the membership function values ​​of the three groups of independent variables are determined. The trapezoidal membership function is used. Taking the membership function of the real-time electricity price as an example, Figure 3 shown.

[0068] exist Figure 3 In the above figure, the four parameters of the membership function are determined by Table 1, which are 0, 0.13, 0.16, and 0.21 ($).

[0069] The membership functions of electric vehicle SOC and external temperature are similar to those of real-time electricity prices. They are both trapezoidal membership functions, and their parameters are determined by Table 1.

[0070] Step 1.4: Determination of fuzzy rules. TSK fuzzy model was used to formulate 27 fuzzy rules in three dimensions.

[0071] The TSK fuzzy model was proposed by Takagi, Sugeno and Kang. It is a type of fuzzy logic system that defines the rules of the fuzzy system with "IF-then" rules. In the electric vehicle demand response model described in the present invention, the TSK fuzzy model can be written as:

[0072]

[0073] Where: x1, x2, x3 are the input variables real-time electricity price, battery SOC and outdoor temperature, respectively. are the fuzzy subsets corresponding to the input variables of the kth rule, namely, incentive electricity price, battery SOC and outdoor temperature, and is the fuzzy connection operator, f k (·) is the value of the kth fuzzy rule, is the fixed parameter of the kth fuzzy rule, and K is the number of fuzzy rules.

[0074] Table 2 Fuzzy rules (a) Fuzzy rules when the temperature is cold

[0075]

[0076] (b) Fuzzy rules when the temperature is comfortable

[0077]

[0078] (c) Fuzzy rules when the temperature is hot

[0079]

[0080] Table 2 shows the fuzzy rules designed for this case. The parameters θ, σ is related to the user's sensitivity to price, battery SoC, and external temperature. In this example, the values ​​are 0.6, 0.7, and 0.8, respectively.

[0081] Step 1.5: Quantitative evaluation of electric vehicle response willingness. According to the fuzzy rule output results of step 4, the response willingness of electric vehicle users can be calculated by the weighted average of the results of each rule, as shown in formula (2):

[0082]

[0083] Where: are the membership functions corresponding to the fuzzy subsets, and λ is the response willingness value of the electric vehicle.

[0084] Using the real-time electricity price, electric vehicle battery SOC, and external temperature data from the CityLearn open source dataset for 8760 hours a year, the response willingness of electric vehicles at each moment is calculated. The visualization results are as follows: Figure 4, where the redder the color, the higher the willingness to respond. (For easy observation, 2000 points are randomly selected for visualization).

[0085] Step 2: Community energy management based on reinforcement learning. The zero-carbon community studied in the present invention includes an electric vehicle cluster. Step 1 completes the quantification of the response willingness of the electric vehicle cluster, and its result is used as data preparation for step 2, which can realize community energy management that fully considers the response willingness of electric vehicles.

[0086] Step 2.1: Data collection. Use sensors or smart meters to obtain real-time information about buildings and electric vehicle loads in the community. For example, real-time electricity price information, household energy storage SOC, electric vehicle battery SOC, electric vehicle charging demand, electric vehicle charging time, photovoltaic system power generation output, outside temperature, solar radiation, building air conditioning load, building hot water load, non-transferable household load, etc.

[0087] The present invention uses data from the CityLearn virtual community, which consists of 17 buildings, each of which is equipped with a battery energy storage system and a solar energy system, and some buildings are equipped with electric vehicle chargers that support different power levels (7.4kW, 11kW, and 22kW).

[0088] The meteorological data used in the present invention comes from the TMY3 file of the Los Angeles Weather Station, which contains hourly temperature, humidity, and solar radiation time series data for a year. The mean temperature is 16.84°C and the quartile range is 5°C.

[0089] The electricity price data used in the present invention comes from the real-time electricity price plan of Southern California Edison Electric Supply Company, and uses hourly electricity price data for 8760 hours a year, with an average of 0.17 kWh / $ and an interquartile range of 0.08 kWh / $.

[0090] Step 2.2: Reinforcement learning algorithm training. Using the data collected in step 2.1, train the reinforcement learning controller.

[0091] The present invention adopts the Soft Actor-Critic (SAC) algorithm based on maximum entropy and policy gradient as a reinforcement learning controller. SAC is a model-free, non-policy-driven RLC algorithm suitable for learning in continuous action space, and has strong self-regulation and exploration capabilities. It is widely used in virtual power plant multi-energy flow scheduling, distribution network economic scheduling, energy system optimization control and microgrid operation management.

[0092] The SAC algorithm architecture is as follows Figure 5As shown in the figure, the SAC algorithm consists of an evaluation network and a target network, each of which contains two sub-networks: Actor and Critic. Based on the experience replay mechanism, the system samples experience data tuples (state, action, reward, next moment state) from the replay buffer for network training.

[0093] During the training process, the Actor of the evaluation network generates continuous actions of the energy storage control strategy in the community based on the current state, and the Critic network evaluates the value of the action. The target network gradually tracks the parameters of the evaluation network through a soft update mechanism (τ = 0.05) to improve the training stability. This algorithm makes full use of historical data and learns the optimal scheduling strategy of energy storage resources in different spatiotemporal scenarios through the dual optimization of the Actor-Critic architecture, providing an effective solution for achieving the energy conservation and emission reduction goals of the community energy management system.

[0094] Taking action entropy into account, the reward function in reinforcement learning can be written as:

[0095]

[0096] in, represents the expectation under the state-action trajectory generated by strategy π, π * represents the optimal strategy, (s t , a t ) represents the state and action pair, ρ π represents the state-action distribution under strategy π. R represents the reward function, α is the temperature parameter used to adjust the degree of exploration. H(π) represents the entropy of the strategy.

[0097] SAC is based on three key elements: actor-criticism architecture, off-policy updating, and entropy maximization. The SAC algorithm consists of three neural networks: actor network, critic network, and target critic network. The corresponding parameters are θ, θ′, and update the network parameters by equation (4).

[0098]

[0099] in, represents the expectation of sampling states from the experience replay buffer D and sampling actions according to the current strategy, J(φ) is the loss function of the Actor, π φ (a i |s i ) is the strategy distribution output by the Actor network, Q0(a i |s i ) is the state-action value function estimated by the Critic network, and α is the temperature parameter used to balance exploration and exploitation.

[0100]

[0101] in, represents the expectation of sampling state-action-next action from the experience replay buffer D, J(θ) is the loss function of Critic, Q θ (a t |s t ) is the value estimate of the current Critic network for the state-action pair, and y is the target value, which is defined by formula (6).

[0102]

[0103] in, represents the expectation of sampling the next action according to the current strategy, r(s t ,a t ) is the immediate reward, γ is the discount factor, Q θ′ is the Q value estimate of the target Critic network, π φ (a t+1 |s t+1 ) is the strategy distribution at the next moment.

[0104] θ′=τθ+(1-τ)θ′ (7)

[0105] Among them, θ′ is the network parameter of the target Critic, τ is the soft update coefficient, and θ is the network parameter of the current Critic.

[0106]

[0107] Among them, J(α) is the loss function of the temperature parameter α, and H0 is the target entropy value, which is used to automatically adjust the balance between exploration and utilization.

[0108] Step 2.2.1: Design of state space and action space. According to the community environment, the state space is designed, which contains two types of information: environmental status (date type, time, direct solar irradiance, real-time carbon emission coefficient, real-time electricity price) and building status (non-transferable load, photovoltaic power generation, energy storage battery charge state, net power consumption, electric vehicle battery charge state); the action space is designed, which includes the charging and discharging control decisions of the building fixed energy storage device and the electric vehicle battery.

[0109] Step 2.2.2: Evaluation index design. In order to evaluate the performance of the reinforcement learning agent, the present invention uses seven evaluation indicators: power consumption (D), electricity cost (C), carbon emissions (G), net power consumption (Z), daily average peak (P), power ramp (R), and 1-load factor (1-L). The smaller the value of these seven indicators, the better the performance. In formula (9)-formula (15), n is the number of time steps in each training set. P, R and (1-L) are community-level evaluation indicators, using the total power consumption of the community per hour E h district The remaining four items are building-level evaluation indicators, using the building's hourly electricity consumption E h building (kWh) calculation.

[0110] The power consumption D represents the total power consumption of the building. The smaller its value is, the smaller the total energy consumption is, as shown in formula (9):

[0111]

[0112] The electricity cost C represents the product of electricity consumption and electricity price, as shown in formula (10), where T h ($ / kWh) is the price of electricity at time h.

[0113]

[0114] Carbon emissions G is obtained by multiplying and adding the electricity consumption and the hourly carbon emission factor, as shown in formula (11).

[0115] Among them, O h is the carbon emission factor at hour h (kgCO 2e / kWh).

[0116]

[0117] Net power consumption Z indicates positive or negative The sum of the values ​​is shown in formula (12):

[0118]

[0119] The average daily peak value P represents the average value of the maximum daily power consumption in the community, as shown in formula (13).

[0120] where d is the index of the day of the year.

[0121]

[0122] The power ramp R represents the absolute value of the difference in community power consumption between two consecutive moments, as shown in formula (14). R represents the smoothness of the load curve in the area. Too high R means that the grid load changes suddenly, which may put pressure on the grid and even cause power outages.

[0123]

[0124] The load factor L is the average ratio of the average monthly electricity consumption of the community to the peak monthly electricity consumption of the community, as shown in formula (15). Where m is the index of the month, and L is between 0 and 1, so the goal is to maximize L or minimize (1-L).

[0125]

[0126] Among them, the superscript district represents community data, 730m+h represents the number of hours counted from the beginning of the mth month, and h in the numerator and summation symbol ranges from 0 to 729, indicating the number of hours in a month (8760 hours in a year / 12=730 hours). The numerator calculates the average hourly electricity consumption of this month, and the denominator represents the peak electricity consumption within this month.

[0127] Step 2.2.3: Reward function design. The present invention adopts a multi-objective combination approach to design a reward function taking into account the seven evaluation indicators described in step 2.2.2, mainly combining the three dimensions of grid benefit reward, user comfort reward, and electric vehicle demand response reward. The overall reward function can be expressed as:

[0128] R t =R g +ω c R c +ω v R EV (16)

[0129] In the formula, R t represents the total reward function, R g represents the grid benefit reward function, R c represents the user comfort reward function, R EV represents the electric vehicle reward function, ω c and ω v are the weight coefficients of user comfort and electric vehicle items respectively.

[0130] The grid benefit reward function can be expressed as:

[0131] R g =-(1+sign(P g )·SOC ess )·ω co C α +ω pP β |+R p +R r (17)

[0132] Where P g is the power taken from the grid, SOC ess is the state of charge of the energy storage system, C is the carbon emission intensity, P is the electricity price, α and β are penalty indexes, ω co and ω p is the weight coefficient of carbon emission and electricity price, R p With R r They are peak-to-valley difference penalty and power climbing penalty respectively.

[0133] The user comfort reward function can be expressed as:

[0134]

[0135] Where, T in is the indoor temperature, T set is the set temperature, ΔT b is the temperature comfort range, and α1 is the penalty index.

[0136] The EV demand response incentive can be expressed as:

[0137] R EV =δ·max(0,-P ev )·sign(P g ) (19)

[0138] Where, δ is the response willingness of the electric vehicle, which can be calculated according to step 1; P ev Charging and discharging power for electric vehicles.

[0139] Table 3 Reward function parameter settings

[0140]

[0141] The reward function parameter settings are shown in Table 3.

[0142] Step 2.2.4: Hyperparameter design of the reinforcement learning agent.

[0143] Table 4 SAC agent hyperparameters

[0144]

[0145] Table 4 shows the hyperparameters of the reinforcement learning agent of the present invention. The neural network of the agent has two hidden layers, each with 256 neurons. The batch size of the strategy update and the replay buffer capacity of the batch sampling are 512 and 100000 respectively. There are 8760 time steps (1 year) in one round. At the beginning of the training, a random exploration of 4380 time steps is first performed. After the exploration, SAC reinforcement learning training is started for 10 rounds. The decay rate of 0.05 means that in the soft update process of the target network, the weight of the new parameter is 5% and the weight of the original parameter is retained at 95%. The discount factor of 0.9 means that when calculating the cumulative return, the return of each future step will decay to 0.9 times the original value compared to the current moment, indicating that the agent will consider short-term and long-term benefits relatively balanced. The learning rate of 0.005 means that when the parameters are updated, the step size of 0.5% is moved in the direction of the gradient each time. In each round, the environment is reset to the initial state, and then the training of the agent is resumed until the last time step is reached again. By exploring different actions, the agent gradually converges to the action combination with the maximum cumulative reward as the number of training rounds increases, and obtains the optimal control strategy. Finally, the trained reinforcement learning agent executes a round of deterministic strategy for evaluation.

[0146] Step 2.2.5: Reference baseline design. To compare the performance of the reinforcement learning controller, a rule-based control (RBC) strategy is designed as a reference baseline.

[0147] The control rules of the zero-carbon community battery energy storage system are as follows: when the net power consumption of the building is greater than 0 and the battery state of charge is greater than 25%, the battery is controlled to discharge at 78% of the maximum power; when the net power consumption of the building is negative, the battery is controlled to charge at 78% of the maximum power to store excess power. In other cases, the battery remains in a silent state.

[0148] The control rules for electric vehicles in zero-carbon communities are as follows: when the state of charge of the electric vehicle battery is lower than 20%, the charging pile is controlled to charge the electric vehicle at maximum power to ensure that the electric vehicle meets travel needs. When the battery state of charge is higher than 80%, the charging pile is controlled to discharge at maximum power for demand response or to support building electricity use. When the battery state of charge is between 20% and 80%, the charging pile does not perform charging and discharging operations.

[0149] Step 2.2.6: Train reinforcement learning strategy. Deploy the SAC strategy in the CityLearn virtual community, use the RBC strategy as the reference baseline, train ten rounds for 8760 hours of data per year, and then deploy the trained agent to the corresponding building, simulate one round, and record and analyze the results. The hardware environment for reinforcement learning training in the present invention is NVIDIA GeForceRTX 306012G, running memory 32G, CPU is Intel i7-13700K, and the software environment is Windows 11, Cuda 11.2.

[0150] The reinforcement learning agent training process is as follows Figure 6 As shown, Figure 6 The serial number above each sub-graph in the figure refers to the building serial number, which is Figure 6 It can be seen that in the training process of each building agent, as the number of training rounds increases, the reward function value first increases continuously, and then remains stable until the end of training. The convergence speed of each building is different. After the fourth round, all buildings in the community converge.

[0151] Step 2.3: Deployment of reinforcement learning strategy. Deploy the reinforcement learning strategy trained in step 2.2.6 and the reference baseline designed in step 2.2.5 in the CityLearn virtual community for comparative analysis. Use the seven evaluation indicators designed in step 2.2.2 for comparison. After calculating the evaluation index results of each building, use the RBC strategy as the reference baseline and the SAC strategy indicator as the relative value. The results are as follows: Figure 7 .

[0152] Depend on Figure 7 It can be seen that the community energy management system under the SAC strategy has established a more reasonable energy flow allocation strategy. Its power consumption (referring to the power purchased from the power grid), electricity charges, and carbon emissions are only 27%, 26%, and 27% of the RBC strategy, respectively, which has significant advantages in energy conservation and emission reduction. The average daily peak power consumption under the SAC strategy is only 50% of that of the RBC strategy, and the 1-load rate is 78% of that of the RBC strategy. By “peak shaving and valley filling”, the “duck curve” problem of distributed photovoltaic system power consumption is alleviated, and the load characteristics are improved. The net power consumption of the SAC strategy (referring to the difference between the power purchased from the power grid and the power fed to the power grid) is 62% of that of the RBC strategy. The SAC intelligent body improves the photovoltaic absorption rate, achieves local energy balance more efficiently, and reduces dependence on the external power grid.

[0153] Aiming at the problems of insufficient quantification of electric vehicle response willingness, poor adaptability to complex environments, and low utilization of regulation resources in community energy management, the present invention proposes a community-level electric vehicle-household energy storage collaborative scheduling method and system. A TSK fuzzy response willingness evaluation model based on electricity price, battery SOC and temperature was established. In the fuzzy rule design stage, 27 fuzzy rules covering multi-dimensional influencing factors were constructed. In the parameter optimization stage, the membership function was solved to achieve accurate quantification of user response willingness. On this basis, in order to achieve the coordinated optimization of energy storage and electric vehicles, a SAC reinforcement learning controller based on the Actor-Critic architecture was constructed. An intelligent agent design method with a state space including environmental state and building state and an action space covering energy storage devices and electric vehicle control decisions was proposed, and the community energy management strategy was optimized based on the maximum entropy framework. The present invention effectively improves the predictability of electric vehicle response capability, and the control strategy has good adaptability. Compared with the rule-based control method, the power consumption, electricity bills and carbon emissions are reduced by about 73%, and the average daily peak is reduced by 50%.

[0154] Embodiment 2

[0155] This embodiment provides a community-level electric vehicle-household energy storage coordinated scheduling system.

[0156] A community-level electric vehicle-household energy storage coordinated dispatching system, comprising:

[0157] The fuzzy processing module is configured to: divide the real-time electricity price, battery SOC and external temperature into several fuzzy subsets, determine the membership function of each fuzzy subset based on the TSK fuzzy mathematical model; introduce price sensitivity parameters, battery state parameters and temperature sensitivity parameters, formulate fuzzy rules for each fuzzy subset in three dimensions, and quantitatively evaluate the response willingness of electric vehicles through fuzzy reasoning of all membership functions;

[0158] The reinforcement learning module is configured as follows: using the SAC algorithm, designing the state space and action space respectively according to the community environment, and constructing an overall reward function combining grid benefit rewards, user comfort rewards and electric vehicle demand response rewards; inputting the quantitative evaluation results into the SAC algorithm to obtain the optimal scheduling strategy for energy storage resources.

[0159] In some embodiments, the real-time electricity price, battery SOC and external temperature are divided into several fuzzy subsets, and the method includes: dividing the real-time electricity price into three fuzzy subsets of low, medium and high, dividing the battery SOC into three fuzzy subsets of insufficient, medium and sufficient, and dividing the external temperature into three fuzzy subsets of cold, comfortable and hot.

[0160] In some embodiments, the method of determining the membership function of each fuzzy subset based on the TSK fuzzy mathematical model includes: determining four key parameters of the membership function based on the TSK fuzzy mathematical model, and establishing the membership function of each fuzzy subset.

[0161] In some embodiments, the method of quantitatively evaluating the response willingness of electric vehicles by fuzzy reasoning of all membership functions includes: calculating the weighted average of all fuzzy rule output results by performing fuzzy reasoning on all membership functions to calculate the response willingness of electric vehicles.

[0162] In some embodiments, the state space includes: environmental state and building state, the environmental state includes: date type, time, direct solar irradiance, real-time carbon emission coefficient and real-time electricity price, the building state includes: non-transferable load, photovoltaic power generation, energy storage battery charge state, net power consumption and electric vehicle battery charge state.

[0163] In some embodiments, the action space and method include: charging and discharging control decisions for building fixed energy storage devices and electric vehicle batteries.

[0164] In some embodiments, the overall reward function is expressed by the following formula:

[0165] R t =R g +ω c R c +ω v R EV

[0166] In the formula, R t represents the total reward function, R g represents the grid benefit reward function, R c represents the user comfort reward function, R EV represents the electric vehicle reward function, ω c and ω v are the weight coefficients of user comfort and electric vehicle items respectively.

[0167] In some embodiments, during the training process of the SAC algorithm, the Actor network of the SAC algorithm is evaluated, continuous actions of the energy storage control strategy in the community are generated based on the current state, and the value of the action is evaluated by the Critic network, and the target network gradually tracks the parameters of the SAC algorithm through a soft update mechanism.

[0168] In some embodiments, during the training process of the SAC algorithm, evaluation indicators are designed to evaluate the performance of the SAC algorithm, and the evaluation indicators include: power consumption, electricity cost, carbon emissions, net power consumption, daily average peak, power ramp and 1-load factor.

[0169] In some embodiments, during the training process of the SAC algorithm, the performance of the SAC algorithm is optimized by adjusting the network parameters or reward function weights of the SAC algorithm.

[0170] In some embodiments, during the training process of the SAC algorithm, hyperparameters and reference baselines are designed for SAC algorithm reinforcement learning and performance evaluation.

[0171] Embodiment 3

[0172] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the community-level electric vehicle-household energy storage coordinated scheduling method as described in the first embodiment above are implemented.

[0173] Embodiment 4

[0174] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the community-level electric vehicle-household energy storage coordinated scheduling method as described in the first embodiment are implemented.

[0175] Embodiment 5

[0176] This embodiment provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the community-level electric vehicle-household energy storage coordinated scheduling method described in the first embodiment.

[0177] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0178] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0179] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0181] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0182] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A community-level electric vehicle-household energy storage coordinated scheduling method, characterized in that: include: The real-time electricity price, battery SOC and external temperature are divided into several fuzzy subsets. Based on the TSK fuzzy mathematical model, the membership function of each fuzzy subset is determined. The price sensitivity parameter, battery state parameter and temperature sensitivity parameter are introduced to formulate fuzzy rules for each fuzzy subset in three dimensions. Through fuzzy reasoning of all membership functions, the response willingness of electric vehicles is quantitatively evaluated. The SAC algorithm is used to design the state space and action space according to the community environment. The overall reward function is constructed by combining grid benefit rewards, user comfort rewards and electric vehicle demand response rewards. The quantitative evaluation results are input into the SAC algorithm to obtain the optimal scheduling strategy for energy storage resources.

2. The community-level electric vehicle-household energy storage coordinated scheduling method according to claim 1 is characterized in that: The method of dividing the real-time electricity price, battery SOC and external temperature into several fuzzy subsets includes: dividing the real-time electricity price into three fuzzy subsets of low, medium and high, dividing the battery SOC into three fuzzy subsets of insufficient, medium and sufficient, and dividing the external temperature into three fuzzy subsets of cold, comfortable and hot.

3. The community-level electric vehicle-household energy storage coordinated scheduling method according to claim 2 is characterized in that: The method for determining the membership function of each fuzzy subset based on the TSK fuzzy mathematical model includes: determining four key parameters of the membership function based on the TSK fuzzy mathematical model, and establishing the membership function of each fuzzy subset.

4. The community-level electric vehicle-household energy storage coordinated scheduling method according to claim 1 is characterized in that: The method of quantitatively evaluating the response willingness of electric vehicles by fuzzy reasoning of all membership functions includes: calculating the weighted average of all fuzzy rule output results by fuzzy reasoning of all membership functions, and calculating the response willingness of electric vehicles.

5. The community-level electric vehicle-household energy storage coordinated scheduling method according to claim 1 is characterized in that: The state space includes: environmental state and building state. The environmental state includes: date type, time, direct solar irradiance, real-time carbon emission coefficient and real-time electricity price. The building state includes: non-transferable load, photovoltaic power generation, energy storage battery charge state, net power consumption and electric vehicle battery charge state. or, The action space,method includes: charging and discharging control decisions of building fixed energy storage devices and electric vehicle batteries; or, The overall reward function is expressed by the following formula: R t =R g +oh c R c +oh v R EV In the formula, R t represents the total reward function, R g represents the grid benefit reward function, R c represents the user comfort reward function, R EV represents the electric vehicle reward function, ω c and ω v are the weight coefficients of user comfort and electric vehicle items respectively.

6. The community-level electric vehicle-household energy storage coordinated scheduling method according to claim 1 is characterized in that: During the training process of the SAC algorithm, the Actor network of the SAC algorithm is evaluated, and the continuous actions of the energy storage control strategy in the community are generated based on the current state. The value of the action is evaluated by the Critic network, and the target network gradually tracks the parameters of the SAC algorithm through the soft update mechanism. or, During the training process of the SAC algorithm, evaluation indicators are designed to evaluate the performance of the SAC algorithm, and the evaluation indicators include: power consumption, power cost, carbon emissions, net power consumption, daily average peak, power ramp, and 1-load factor; or, During the training process of the SAC algorithm, the performance of the SAC algorithm is optimized by adjusting the network parameters or reward function weights of the SAC algorithm; or, During the training process of the SAC algorithm, hyperparameters and reference baselines are designed for SAC algorithm reinforcement learning and performance evaluation.

7. A community-level electric vehicle-household energy storage coordinated dispatching system, characterized in that: include: The fuzzy processing module is configured to: divide the real-time electricity price, battery SOC and external temperature into several fuzzy subsets, determine the membership function of each fuzzy subset based on the TSK fuzzy mathematical model; introduce price sensitivity parameters, battery state parameters and temperature sensitivity parameters, formulate fuzzy rules for each fuzzy subset in three dimensions, and quantitatively evaluate the response willingness of electric vehicles through fuzzy reasoning of all membership functions; The reinforcement learning module is configured as follows: using the SAC algorithm, designing the state space and action space respectively according to the community environment, and constructing an overall reward function combining grid benefit rewards, user comfort rewards and electric vehicle demand response rewards; inputting the quantitative evaluation results into the SAC algorithm to obtain the optimal scheduling strategy for energy storage resources.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the community-level electric vehicle-household energy storage coordinated scheduling method as described in any one of claims 1-6 are implemented.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the community-level electric vehicle-household energy storage coordinated scheduling method as described in any one of claims 1-6 are implemented.

10. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, the steps in the community-level electric vehicle-household energy storage coordinated scheduling method as described in any one of claims 1-6 are implemented.

Images