Slow charging and quick discharging method of storage and charging integrated charging pile

Abstract

The invention discloses a slow charging and quick discharging method of a storage and charging integrated charging pile. Comprising the following steps: S1, predicting the charging demand of the electric vehicle through obtained power grid load data, energy storage system state information and other external factors; s2, predicting the state of the energy storage system through the acquired information such as the charge state, the temperature condition and the health degree of the battery; s3, constructing a slow-charging and quick-discharging scheduling strategy based on the prediction result of the charging demand of the electric vehicle and the prediction result of the state of the energy storage system in combination with factors such as an electricity price policy and battery life; and S4, monitoring the load condition of the power grid, the use condition of the charging pile and the state information of the energy storage system in real time, and optimizing and adjusting the scheduling strategy of slow charging and quick discharging. By optimizing the charging and discharging processes, efficient utilization of energy, stability of the power grid load and prolonging of the service life of the battery are achieved, and the charging and discharging strategies can be flexibly adjusted according to the power grid load and electricity price changes.

Description

Technical Field

[0001] The present invention relates to the technical field of decision scheduling and reinforcement learning, and in particular to a slow charging and fast discharging method of a storage-charging integrated charging pile. Background Art

[0002] With the popularity of electric vehicles and the increasing load on the power grid, the importance of integrated storage and charging piles, as a device that combines energy storage and charging functions, has become increasingly prominent. However, existing integrated storage and charging piles often lack efficient scheduling strategies during the charging and discharging process, resulting in low energy utilization efficiency, large fluctuations in the power grid load, and adverse effects on battery life. Therefore, developing an efficient scheduling strategy that can achieve slow charging and fast discharging is of great significance to improving the performance and benefits of integrated storage and charging piles.

[0003] The slow charging and fast discharging scheduling strategy mainly refers to using a slower charging speed during the charging process and achieving faster energy release during discharge or energy output. This strategy is usually used in energy storage systems, especially integrated storage and charging piles, to optimize grid load, improve system efficiency, and meet the fast charging needs of electric vehicles. The following are several common slow charging and fast discharging scheduling strategies and their related algorithm formulas:

[0004] 1. Dispatch strategy based on load forecasting:

[0005] Algorithm formula: This strategy first predicts the changing trend of power grid load through historical data and prediction algorithms (such as time series analysis, machine learning, etc.).

[0006] Dispatching decision: When the grid load is low, slow charging is performed to store excess energy; when the grid load is peak or the electric vehicle charging demand is peak, fast discharge is performed to provide electricity for the grid or electric vehicles.

[0007] Existing deficiencies: The accuracy of load forecasting is affected by many factors, such as weather changes, holidays, emergencies, etc. These uncertainties may lead to large deviations in the forecast results, thus affecting the effectiveness of the dispatch strategy. Even if the forecast is accurate, the dispatch strategy based on the forecast will take a certain amount of time to implement the dispatch, which may not be flexible enough when facing rapidly changing grid loads or electric vehicle charging needs.

[0008] 2. Scheduling strategy based on energy management:

[0009] Algorithm formula: Dynamically adjust the charging and discharging rates by real-time monitoring of the state of the energy storage system (such as SOC, i.e. state of charge), the charging demand of electric vehicles and the state of the power grid.

[0010] Scheduling decision: When the energy storage system SOC is low and the grid load allows, slow charging is performed; when the SOC is high and the grid or electric vehicles need energy, fast discharge is performed.

[0011] Disadvantages: Real-time monitoring of the energy storage system status and electric vehicle charging needs, as well as dynamic adjustment of charging and discharging rates, requires complex control systems and fast data processing capabilities, which increases the complexity and cost of the system. This strategy is highly dependent on the performance of the energy storage system and the charging behavior of electric vehicles. If these factors change, the scheduling strategy may need to be adjusted frequently, increasing the difficulty of management.

[0012] 3. Scheduling strategy based on cost optimization:

[0013] Algorithm formula: Considering factors such as electricity price, charging cost, and discharge benefit, optimization algorithms (such as linear programming, dynamic programming, etc.) are used to minimize charging cost and maximize discharge benefit.

[0014] Scheduling decisions: slow charging during low electricity price periods to store low-cost electricity; fast discharge during high electricity price periods or when electric vehicle charging demand peaks to obtain higher economic returns.

[0015] Existing shortcomings: Excessive pursuit of cost optimization may lead to increased fluctuations in grid load and have an adverse impact on grid stability and security. It is necessary to find a balance between economy and system performance.

[0016] 4. Scheduling strategy based on battery life optimization:

[0017] Algorithm formula: Considering factors such as the number of battery charge and discharge cycles and the number of deep discharges, the algorithm is optimized to balance the battery's charging and discharging speeds to extend battery life.

[0018] Scheduling decision: Avoid frequent deep discharges and try to keep the battery in the medium SOC range for slow charging and fast discharge.

[0019] Disadvantages: In order to extend battery life, this strategy may tend to adopt a more conservative charging and discharging strategy, which may sacrifice certain system efficiency and energy utilization. Summary of the invention

[0020] The purpose of the present invention is to overcome the problems existing in the above-mentioned prior art and to provide a slow charging and fast discharging method for an integrated storage and charging charging pile, aiming to achieve efficient utilization of energy, reduce grid load fluctuations, and extend battery life by optimizing the charging and discharging process.

[0021] In order to achieve the above object, the present invention provides a slow charging and fast discharging method of a storage-charging integrated charging pile, comprising the following steps:

[0022] S1. Predicting the charging demand of electric vehicles by acquiring grid load data, energy storage system status information and other external factors;

[0023] S2, predicting the state of the energy storage system by obtaining information such as the state of charge, temperature and health of the battery;

[0024] S3. Based on the prediction results of electric vehicle charging demand and energy storage system status, and combined with factors such as electricity price policy and battery life, a slow charging and fast discharging scheduling strategy is constructed;

[0025] S4. Real-time monitoring of the grid load, charging pile usage and energy storage system status information to optimize and adjust the slow charging and fast discharging scheduling strategy.

[0026] Preferably, step S1 specifically includes the following steps:

[0027] S101. Establish a prediction model based on long short-term memory network based on deep learning;

[0028] S102, normalize the acquired data, and the conversion formula is:

[0029]

[0030] Where X is the original data, X min and X max are the minimum and maximum values ​​in the data set, respectively;

[0031] S103, generating a series of data sets by means of sliding window processing, and dividing the data sets into a training set and a test set;

[0032] S104, evaluating the prediction model using a cross-validation method;

[0033] S105, optimizing the parameters of the prediction model by using a back propagation algorithm and a gradient descent method to minimize the prediction error.

[0034] Preferably, step S2 specifically includes the following steps:

[0035] S201, using a filtering method to remove interference noise in the data;

[0036] S202: Use a state estimation method to perform data fusion to predict the state of the energy storage system.

[0037] Preferably, the state estimation method is to estimate the remaining power and health status of the energy storage system.

[0038] Preferably, the remaining power is estimated using the Ampere integral method, which is expressed as follows:

[0039]

[0040] Where Q is the charge change, I(t) is the current that changes with time, and t1 and t2 are the time intervals of integration.

[0041] Preferably, the health status is estimated using an equivalent circuit model, and the specific steps are:

[0042] Establish an equivalent circuit model and define the model parameters: internal resistance R, open circuit voltage V OC , capacity C;

[0043] According to the real-time measured voltage V, current I and temperature T, combined with the model parameters, the state equation and observation equation of the battery are established.

[0044] The state equation describes the change of the internal state of the battery (such as SOC) over time, which is related to the current I and the model parameters:

[0045] x k+1 =f(x k ,u k )+w k

[0046] Among them, x k is the state vector, u k is the input vector, w k is the process noise, f is the state transfer function,

[0047] Observation equation: describes the relationship between the external observables and the internal state of the battery:

[0048] z k =h(x k )+v k

[0049] where z k is the observation vector, x k is the observation noise, h is the observation function;

[0050] Using a Kalman filter in combination with the state equation and the observation equation, the model parameters are estimated and updated online;

[0051] The health status is calculated using the estimated model parameters.

[0052] Preferably, step S3 specifically includes the following steps:

[0053] S301. Define the scheduling strategy decision type: the action space is A, which represents all possible action sets:

[0054] A={a1,a2,…,an}

[0055] where a i Indicates different operations of the charging pile, including adjusting the charging mode and adjusting the fast-discharge power;

[0056] S302, define the observation information S that can be obtained by the scheduling strategy, including the current state of charge, temperature, health of the storage and charging system, whether there is a charging demand at the current time, the expected charging time, the charging demand forecast in the future period, the power grid electricity price in the future period, etc., that is,

[0057] S={s1,s2,…,s N};

[0058] S303, constructing a reward function;

[0059] S304. Optimize the scheduling strategy using a reinforcement learning algorithm based on a value function.

[0060] Preferably, the reward function specifically includes:

[0061] Reward function based on battery health:

[0062] R health (s,a) = Func (battery health, charge and discharge power, charge and discharge time)

[0063] Where Func represents an abstract function;

[0064] Reward function based on charging efficiency:

[0065] R efficiency (s,a) = battery charging efficiency × charging power;

[0066] Cost-based reward function:

[0067] R cost (s,a) = discharge benefit - charging cost - maintenance cost;

[0068] Reward function based on user experience:

[0069] R userexperience (s,a)=-user waiting time;

[0070] The overall reward function:

[0071] R total (s,a)=w1×R health (s,a)+w2×R efficiency (s,a)+

[0072] w3×R cost (s,a)+w4×R user_experience(s,a)

[0073] Among them, w1, w2, w3, and w4 are coefficients for weighing various factors.

[0074] Preferably, the reinforcement learning algorithm is a Q-learning algorithm, and the update formula is:

[0075] Q(s,a)←Q(s,a)+α[r+γmax a′ Q(s′,a′)-Q(s,a)]

[0076] Where α is the learning rate, γ is the discount factor, r is the immediate reward, s′ is the next state, and a′ is the next action.

[0077] Compared with the prior art, the present invention has the following beneficial effects:

[0078] 1. The present invention can flexibly adjust the charging and discharging strategies according to the changes in grid load and electricity prices, thereby achieving efficient use of energy and reducing costs. For charging pile operators, the economic benefits can be significantly improved by optimizing charging strategies, improving equipment utilization and reducing maintenance costs. At the same time, as an important part of smart grids and smart cities, smart charging piles have also attracted government subsidies and private investment, bringing long-term and stable returns to investors.

[0079] 2. The present invention avoids excessive damage to the battery and prolongs the battery life by precisely controlling the charging and discharging process. By precisely controlling the charging and discharging process, energy waste and carbon emissions are reduced, especially charging during the low electricity price period, which not only reduces the charging cost of users, but also indirectly promotes energy conservation and emission reduction of the power grid, and has a positive effect on alleviating the urban heat island effect and improving air quality.

[0080] 3. The present invention can be combined with technologies such as smart grid and Internet of Things to realize functions such as remote monitoring and fault warning, and improve the management and maintenance efficiency of charging piles. The intelligent charging and discharging strategy of the present invention can serve as a virtual energy storage unit of the power grid, releasing electric energy when the power grid load is peak and absorbing electric energy when the load is low, effectively alleviating the peak-to-valley difference of the power grid, enhancing the elasticity and stability of the power grid, and reducing the risk of power outages. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 This is a framework diagram of the training scheduling strategy for the reinforcement learning algorithm based on the value function provided by the present invention. DETAILED DESCRIPTION

[0082] The innovative features of the present invention are specifically embodied in the following aspects:

[0083] First, the accuracy of charging demand forecasting is improved by combining real-time and historical data, as well as external factors such as weather and traffic flow, through the deep learning long short-term memory network (LSTM) technology. At the same time, in terms of energy storage system state prediction, a variety of sensors and filtering technologies are used to collect and process data in real time, combined with state estimation methods such as Kalman filters, to achieve accurate prediction of battery state of charge (SOC) and health state (SOH), thereby enhancing the reliability of energy storage system state prediction.

[0084] Secondly, the invention designs a set of intelligent slow-charging and fast-discharging scheduling strategies. This strategy models the scheduling problem as a Markov decision process (MDP) and optimizes it through reinforcement learning algorithms (such as deep Q network DQN), realizing dynamic adjustment based on real-time information such as grid load, electricity price, charging demand and energy storage system status. A comprehensive reward function is designed to comprehensively consider multiple factors such as battery health, charging efficiency, cost and user experience. By weighing these factors to formulate the optimal scheduling strategy, it not only ensures the long-term performance of the system, but also improves the user experience.

[0085] Furthermore, the invention introduces a real-time feedback and dynamic optimization mechanism. By comprehensively monitoring the grid load, charging pile usage and energy storage system status information, the slow charging and fast discharging scheduling strategy is dynamically optimized and adjusted to ensure that the system can adapt to complex and changing environmental conditions and improve the stability and reliability of the system. At the same time, the introduction of the user feedback mechanism incorporates user experience factors such as user waiting time into the optimization range of the scheduling strategy, further improving user satisfaction.

[0086] Finally, the invention achieves a deep integration of technology and system. It deeply integrates advanced technologies such as smart grid, Internet of Things, deep learning and reinforcement learning to build a highly intelligent storage and charging integrated charging pile system. This system not only realizes functions such as remote monitoring, fault warning, and automatic scheduling, significantly improving the management and maintenance efficiency of charging piles; and through deep integration with smart grids, it not only enhances the stability and reliability of the power grid, but also promotes the consumption of renewable energy and the realization of energy conservation and emission reduction goals.

[0087] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0088] The present invention comprehensively considers the variable factors involved in the slow charging and fast discharging decision, designs a comprehensive reward function, adopts reinforcement learning to construct decision actions and observation space, and provides a slow charging and fast discharging method for a storage-charging integrated charging pile through the reinforcement learning algorithm.

[0089] The present invention proposes a method for slow charging and fast discharging of a charging pile with integrated storage and charging, such as Figure 1 As shown, the specific steps are as follows:

[0090] Step S1: predicting the charging demand of electric vehicles by obtaining grid load data, energy storage system status information and other external factors.

[0091] Specifically, various sensors and monitoring devices are used to obtain real-time grid load data, electric vehicle charging demand information, and energy storage system status information, including but not limited to the use of charging piles, battery status (such as SOC), vehicle charging demand, charging power, etc. At the same time, external data such as weather conditions, traffic flow and other related factors are also integrated to improve the accuracy of charging demand prediction.

[0092] Based on the data acquired in real time, an advanced prediction model building method is adopted. Among them, for the charging demand of electric vehicles, the present invention adopts the long short-term memory network (LSTM) technology based on deep learning to establish a prediction model. These models can be trained based on historical data and real-time data, and can accurately predict future charging demand on different time scales (hours, days, weeks, etc.).

[0093] The main technical means involved in the long short-term memory network model based on deep learning include data preprocessing, model construction, model training and model evaluation and optimization.

[0094] First, the time series data needs to be converted into a format suitable for LSTM input, including normalization and sliding window processing. Normalization is a technique that scales the data so that it falls into a small specific interval (usually [0,1] or [-1,1]). The purpose of this is to eliminate the impact of different dimensions on data analysis and improve the convergence speed and prediction accuracy of the model.

[0095] The conversion formula is:

[0096]

[0097] Where X is the original data, X min and X max are the minimum and maximum values ​​in the data set, respectively.

[0098] Features such as historical charge, weather conditions (such as temperature, humidity, etc., if they are used as input features), and time factors (such as encoded values ​​such as hours, days, weeks, etc.) are normalized so that these features are numerically of the same order of magnitude to avoid numerical problems during model training.

[0099] Sliding window processing slides a fixed-size window over the original data and processes the data in each window as a sample. This can capture the local characteristics of the data in the time series and generate a sample set suitable for model training.

[0100] Determine the window size and step size. The window size determines the number of time points contained in each sample, and the step size determines the distance the window slides on the data. Use the sliding window to slide on the original data to generate a series of samples. Each sample includes the data point in the window and the corresponding label (charging demand).

[0101] Through sliding window processing, the original continuous time series data can be converted into a series of samples, each of which contains charging data and other related information (such as weather, time, etc.) in the past period of time, as well as the charging demand at a certain point in the future as a label. In this way, deep learning models (such as LSTM) can use these samples for training to learn the relationship between historical data and future charging demand.

[0102] The second step is to divide the data set, dividing all the data into training set and test set in the ratio of 80% and 20% respectively. Use the cross-validation method to evaluate the training model and avoid using the validation set. Specifically, in cross-validation, the data set is divided into K smaller subsets (called "folds"). Then, the model is trained and tested multiple times on these subsets, using one subset as the test set and the remaining K-1 subsets as the training set each time. This process is repeated until each subset has been evaluated as a test set. Finally, the evaluation indicators on all test sets are averaged to obtain a comprehensive evaluation of the model performance.

[0103] In model training, back propagation algorithm and gradient descent technique are used to optimize the parameters of the model to minimize the prediction error.

[0104] The backpropagation algorithm works by calculating the gradient of the loss function with respect to the network weights. The core idea of ​​this algorithm is to backpropagate the error of the loss function with respect to the output layer back to each layer of the network through the chain rule, so as to update the weights of each layer. It mainly includes four steps: 1. Calculate the output of the input data after passing through the neural network. 2. Use loss functions (such as mean square error, cross entropy, etc.) to calculate the difference between the network output and the actual label. 3. Calculate the gradient of the loss function with respect to the network output (that is, the derivative of the loss with respect to the output). Using the chain rule, the gradient is backpropagated to each layer of the network, and the gradient of the loss function with respect to the parameters of each layer is calculated. 4. Use the gradient descent method to update the weights of the network to reduce the loss.

[0105] Gradient descent is used to find the local minimum of a function. In model training, this function is usually a loss function, and the parameters are the weights of the model. Gradient descent reduces the loss by updating the weights in the opposite direction of the gradient of the loss function with respect to the weights. It mainly consists of two steps: 1. For the current weights of the model, calculate the gradient of the loss function with respect to these weights. 2. Update the weights in the opposite direction of the gradient by a certain step size (learning rate).

[0106] The back propagation algorithm is used alternately to calculate the gradient, and the gradient descent method is used to update the model's weights. This process is repeated until a certain stopping condition is met (such as reaching a preset number of iterations, the loss value is less than a certain threshold, or the performance on the validation set no longer improves). In this way, the model can gradually learn the patterns in the data and reduce the prediction error.

[0107] In the process of establishing the prediction model, feature engineering and model optimization were also carried out to improve the performance and stability of the prediction model. By extracting and selecting data features, the relevant factors and trends of charging demand can be better captured. At the same time, the model was optimized using methods such as cross-validation and hyperparameter tuning to further improve the prediction accuracy and generalization ability.

[0108] Step S2: predicting the state of the energy storage system by acquiring information such as the state of charge, temperature, and health of the battery.

[0109] Specifically, multiple sensors are used for real-time data collection, including but not limited to battery SOC sensors, temperature sensors, health monitoring devices, etc. These sensors can monitor and record various state parameters of the energy storage system in real time, such as the battery's state of charge, temperature, health and other important indicators.

[0110] After data collection, filtering methods are used to remove interference noise in the data. Through filtering technology, noise interference in the data can be removed and the accuracy and stability of the data can be improved.

[0111] Then, the state estimation method is used to perform data fusion to predict the state of the energy storage system. The state estimation technology can be used to more accurately estimate and judge the state of the energy storage system by fusing multiple sensor data. The state estimation method is to estimate the remaining power and health status of the energy storage system.

[0112] Estimation of remaining power: The remaining power (State of Charge, SOC) is the ratio of the current available power of the battery to its total capacity, indicating the charging state of the battery. The Ampere integration method is used to continuously monitor the charge and discharge current of the battery and integrate the time to calculate the change in power. The core of this method lies in the product of current and time, that is, ampere-hour (Ah), which directly reflects the amount of power charged or discharged by the battery in a certain period of time.

[0113] Specifically, the calculation formula of Abe's integral method can be expressed as:

[0114]

[0115] Where Q is the charge change, I(t) is the current that changes with time, and t1 and t2 are the time intervals of integration.

[0116] Estimation of health status: The health status (SOH) is a key parameter to measure the degree of battery aging. The equivalent circuit model is used to simulate the internal resistance, capacitance and other circuit components of the battery to reflect the working status of the battery. In SOH estimation, the parameters of these components (such as internal resistance, capacity, etc.) will change as the battery ages. Therefore, SOH can be estimated by monitoring the changes in these parameters.

[0117] The health status is estimated using an equivalent circuit model, and the specific steps are:

[0118] 1. Establish an equivalent circuit model and define model parameters (such as internal resistance R, open circuit voltage V OC , capacity C, etc.).

[0119] 2. According to the real-time measured voltage V, current I and temperature T, combined with the model parameters, the state equation and observation equation of the battery are established.

[0120] State equation: describes the change of the internal state of the battery (such as SOC) over time, which is usually related to the current I and model parameters (such as capacity C).

[0121] x k+1 =f(x k ,u k )+w k

[0122] Among them, x kis the state vector, u k is the input vector, w k is the process noise, and f is the state transfer function.

[0123] Observation equation: describes the relationship between the external observable quantity of the battery (such as voltage V) and the internal state.

[0124] z k =h(x k )+v k

[0125] where z k is the observation vector, x k is the observation noise, and h is the observation function.

[0126] 3. Use Kalman filter or similar algorithm to estimate and update model parameters online based on state equation and observation equation.

[0127] 4. Using the estimated model parameters (especially those related to capacity), calculate SOH.

[0128] Step S3: construct a slow charging and fast discharging scheduling strategy based on the prediction results of the electric vehicle charging demand and the energy storage system status prediction results and in combination with factors such as electricity price policy and battery life.

[0129] Specifically, the present invention models the problem as a Markov decision process (MDP), in which the agent selects actions (charging or discharging speed) according to the current state of the environment (grid status, electricity price, charging demand, etc.) to maximize long-term rewards (such as cost savings, grid stability, etc.), and finally obtains the optimal scheduling strategy through continuous interaction with the environment and training.

[0130] First, define the decision type of the decision system: the action space is A, which represents the set of all possible actions.

[0131] A={a1,a2,…,a n}

[0132] where a i Indicates different operations of the charging pile, including adjusting the charging mode and adjusting the fast-discharge power. For example, the starting power is P in Slow charging, end slow charging, start power P out Fast discharge, direct discharge to the grid, etc.

[0133] Define the observation information S that the decision-making system can obtain, including the current state of charge, temperature, health of the storage and charging system, whether there is a charging demand at the current moment, the expected charging time, the charging demand forecast in the future period, the power grid electricity price in the future period, etc., that is,

[0134] S={s1,s2,…,s N}

[0135] Construct a reward function. The design of the reward function takes into account multiple factors, such as charging efficiency, cost, and user experience. The reward function specifically includes:

[0136] Reward function based on battery health: Considering the impact of battery life on the long-term performance of the system, a reward function is designed to maximize battery health and encourage the selection of charging and discharging power so that the battery is in a better working state during the charging and discharging process;

[0137] R health (s,a) = Func (battery health, charge and discharge power, charge and discharge time)

[0138] Where Func represents an abstract function.

[0139] Reward function based on charging efficiency: During the charging process, the size of the charging power directly affects the charging efficiency. Therefore, a reward function can be designed to encourage the selection of an efficient charging strategy:

[0140] R efficiency (s,a)=battery charging efficiency×charging power

[0141] Cost-based reward function: The operating cost of the charging pile includes electricity cost and equipment maintenance cost, so the reward function is designed to minimize the total cost of the system:

[0142] R cost (s,a)=discharge benefit-charge cost-maintenance cost

[0143] Reward function based on user experience: User waiting time is one of the important indicators to measure user experience. Therefore, a reward function can be designed to minimize user waiting time:

[0144] R user_experience (s,a)=-user waiting time

[0145] The overall reward function:

[0146] R total (s,a)=w1×R health (s,a)+w2×R efficiency (s,a)+w3

[0147] ×R cost (s,a)+w4×R user_experience (s,a)

[0148] Among them, w1, w2, w3, and w4 are coefficients for weighing various factors.

[0149] The scheduling strategy is optimized by using a reinforcement learning algorithm based on a value function. The present invention uses a reinforcement learning algorithm based on a value function, such as a deep Q network (DQN) as the decision algorithm of the agent system. The algorithm continuously optimizes the slow charging and fast discharging strategy of the charging pile by updating the value function. The reinforcement learning algorithm optimizes the decision strategy by continuously updating the value function. For example, using the Q-learning algorithm, the update formula is

[0150] Q(s,a)←Q(s,a)+α[r+γmax a′ Q(s′,a′)-Q(s,a)]

[0151] Where α is the learning rate, γ is the discount factor, r is the immediate reward, s′ is the next state, and a′ is the next action.

[0152] The process of training the slow charging and fast discharging decision of the integrated charging pile through reinforcement learning can be divided into the following steps:

[0153] Step 1: Model the environment of the integrated charging pile system and define the state space, action space, reward function, etc. According to the characteristics and requirements of the system, determine the state representation method, feasible actions, and reward calculation method.

[0154] Step 2: Initialize the parameters required by the reinforcement learning algorithm, including the initial estimate of the value function, the initial weights of the policy network, etc. These parameters can be initialized based on experience or randomly.

[0155] Step 3: The agent system interacts with the environment, selects actions based on the current state, and observes the feedback information (reward and next state) from the environment. Through continuous interaction, training data is accumulated to update the value function and policy network.

[0156] Step 4: Using the collected data, use a reinforcement learning algorithm to update the value function and policy network. Depending on the algorithm selected, you can use a value function-based method (such as Q-learning) or a policy-based method (such as the policy gradient method) to update the parameters.

[0157] Step 5: During the training process, it is necessary to tune parameters and optimize the training strategy according to the actual situation. This may include adjusting hyperparameters such as learning rate, discount factor, reward function weight, and selecting the optimization algorithm for the value function approximator or policy network. Figure 1 shown.

[0158] Step 6: After completing a certain number of training iterations, evaluate the trained strategy by interacting with the real environment or testing it in a simulation environment to evaluate the performance and generalization ability of the strategy. Adjust and improve the strategy based on the evaluation results.

[0159] Step S4: real-time monitoring of the grid load, charging pile usage and energy storage system status information to optimize and adjust the slow charging and fast discharging scheduling strategy.

[0160] Specifically, the present invention designs a comprehensive feedback mechanism. By collecting and analyzing charging pile operation data, battery charging and discharging efficiency, user feedback and other information, the system can promptly discover potential problems and optimization space, and provide corresponding feedback and suggestions.

[0161] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. A slow charging and fast discharging method for a charging pile with integrated storage and charging, It is characterized in that The following steps are involved: S1. Predicting the charging demand of electric vehicles by acquiring grid load data, energy storage system status information and other external factors; S2, predicting the state of the energy storage system by obtaining information such as the state of charge, temperature and health of the battery; S3. Based on the prediction results of electric vehicle charging demand and energy storage system status, and combined with factors such as electricity price policy and battery life, a slow charging and fast discharging scheduling strategy is constructed; S4. Real-time monitoring of the grid load, charging pile usage and energy storage system status information to optimize and adjust the slow charging and fast discharging scheduling strategy.

2. A slow charging and fast discharging method for a storage-charging integrated charging pile according to claim 1, It is characterized in that Step S1 specifically includes the following steps: S101. Establish a prediction model based on long short-term memory network based on deep learning; S102, normalize the acquired data, and the conversion formula is: Where X is the original data, X min and X max are the minimum and maximum values ​​in the data set, respectively; S103, generating a series of data sets by means of sliding window processing, and dividing the data sets into a training set and a test set; S104, evaluating the prediction model using a cross-validation method; S105, optimizing the parameters of the prediction model by using a back propagation algorithm and a gradient descent method to minimize the prediction error.

3. A slow charging and fast discharging method for a charging pile with integrated storage and charging according to claim 1, It is characterized in that Step S2 specifically includes the following steps: S201, using a filtering method to remove interference noise in the data; S202: Use a state estimation method to perform data fusion to predict the state of the energy storage system.

4. A slow charging and fast discharging method for a charging pile with integrated storage and charging according to claim 3, It is characterized in that The state estimation method is to estimate the remaining power and health state of the energy storage system.

5. A slow charging and fast discharging method for a storage-charging integrated charging pile according to claim 4, It is characterized in that The remaining power is estimated by using the Ampere integration method, which is expressed by the formula: Where Q is the charge change, I(t) is the current that changes with time, and t 1 and t 2 is the time interval of integration.

6. A slow charging and fast discharging method for a storage-charging integrated charging pile according to claim 5, It is characterized in that The health status is estimated using an equivalent circuit model, and the specific steps are: Establish an equivalent circuit model and define the model parameters: internal resistance R, open circuit voltage V OC , capacity C; According to the real-time measured voltage V, current I and temperature T, combined with the model parameters, the state equation and observation equation of the battery are established. The state equation describes the change of the internal state of the battery (such as SOC) over time, which is related to the current I and the model parameters: x k+1 =f(x k ,u k )+w k Among them, x k is the state vector, u k is the input vector, w k is the process noise, f is the state transfer function, Observation equation: describes the relationship between the external observables and the internal state of the battery: z k =h(x k )+v k where z k is the observation vector, x k is the observation noise, h is the observation function; Using a Kalman filter in combination with the state equation and the observation equation, the model parameters are estimated and updated online; The health status is calculated using the estimated model parameters.

7. A slow charging and fast discharging method for a charging pile with integrated storage and charging according to claim 6, It is characterized in that Step S3 specifically includes the following steps: S301. Define the scheduling strategy decision type: the action space is A, which represents all possible action sets: A={a 1 ,a 2 ,…,a n } where a i Indicates different operations of the charging pile, including adjusting the charging mode and adjusting the fast-discharge power; S302, define the observation information S that can be obtained by the scheduling strategy, including the current state of charge, temperature, health of the storage and charging system, whether there is a charging demand at the current time, the expected charging time, the charging demand forecast in the future period, the power grid electricity price in the future period, etc., that is, S={s 1 ,s 2 ,…,s N }; S303, constructing a reward function; S304. Optimize the scheduling strategy using a reinforcement learning algorithm based on a value function.

8. A slow charging and fast discharging method for a charging pile with integrated storage and charging according to claim 7, It is characterized in that The reward function specifically includes: Reward function based on battery health: R health (s,a) = Func (battery health, charge and discharge power, charge and discharge time) Where Func represents an abstract function; Reward function based on charging efficiency: R efficiency (s,a) = battery charging efficiency × charging power; Cost-based reward function: R cost (s,a) = discharge benefit - charging cost - maintenance cost; Reward function based on user experience: The overall reward function: R total (s,a)=w 1 ×R health (s,a)+w 2 ×R efficiency (s,a)+ w 3 ×R cost (s,a)+w 4 ×R user_experience (s,a) where w 1 ,w 2 ,w 3 ,w 4 It is a coefficient that weighs various factors.

9. A slow charging and fast discharging method for a storage-charging integrated charging pile according to claim 8, It is characterized in that The reinforcement learning algorithm is the Q-learning algorithm, and the update formula is: Q(s,a)←Q(s,a)+α[r+γmax a′ Q(s′,a′)-Q(s,a)] Where α is the learning rate, γ is the discount factor, r is the immediate reward, s′ is the next state, and a′ is the next action.

Images