Self-adaptive efficient charging and discharging control method, device and system of intelligent energy storage system

Abstract

The invention provides a self-adaptive efficient charging and discharging control method, device and system for an intelligent energy storage system, and relates to the technical field of electric energy storage systems. The method comprises the following steps: collecting multi-dimensional data; distributing a sector for each control factor in the multi-dimensional data in a preset feature sphere, determining a time-varying feature value based on a historical sampling data sequence of the control factors and current sampling data, and determining the height of the sector in combination with the initial radius of the feature sphere; determining a contribution factor of each control factor based on the fan-shaped height dispersion degree and the time-varying characteristic deviation degree of each control factor, and determining a weight factor of each control factor based on a target time sequence of the contribution factors; determining a reward and punishment coefficient based on the contribution factor and the weight factor of each control factor, and updating an income reference value of each execution action of the energy storage equipment based on the reward and punishment coefficient; and determining the charging and discharging state of the energy storage equipment based on the updated profit value. According to the invention, self-adaptive adjustment of charging and discharging of the energy storage equipment can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric energy storage systems, in particular to an adaptive and efficient charging and discharging control method, device and system for an intelligent energy storage system. BACKGROUND

[0002] With the continuous increase of intermittent new energy such as wind power and photovoltaic in the power grid, the power grid faces challenges such as intensified power fluctuation and insufficient frequency regulation capability. Traditional energy storage systems are difficult to meet the dynamic balance demand in the high proportion of renewable energy scenarios due to fixed charging and discharging strategy, slow response speed, and efficiency decay with operating conditions.

[0003] In order to solve the above problems, the related technology usually uses Q-learning algorithm to explore and optimize the charging and discharging strategy in the dynamically changing operating environment. However, the training and application of the above Q-learning algorithm model depend on the environment feedback data, while the external factors such as power grid demand, renewable energy supply and electricity price have significant time-varying, randomness and uncertainty, which leads to the phenomenon of distribution drift of statistical characteristics of training data of Q-learning algorithm model changing with time. The continuously changing distribution may lead to the failure of convergence of the charging and discharging strategy, and frequent adjustment of algorithm parameters will reduce the stability, and the charging and discharging state of the energy storage device cannot be adaptively adjusted. SUMMARY

[0004] In order to solve the problem that the training data of the Q-learning algorithm model of the related technology is prone to the phenomenon of distribution drift of statistical characteristics changing with time, the continuously changing distribution may lead to the failure of convergence of the charging and discharging strategy, frequent adjustment of algorithm parameters will reduce the stability, and the charging and discharging state of the energy storage device cannot be adaptively adjusted, the present application provides an adaptive and efficient charging and discharging control method for an intelligent energy storage system, and the technical solution is as follows: Collecting multi-dimensional data; the multi-dimensional data includes control factors of multiple dimensions affecting the charging and discharging of the energy storage device; Assigning a sector to each of the control factors in a pre-set characteristic sphere, determining a time-varying characteristic value based on a historical sampling data sequence and a current sampling data of the control factor, and determining a sector height corresponding to the time-varying characteristic value and an initial radius of the characteristic sphere; Determining a contribution factor of each of the control factors based on the discrete degree of the sector height and the time-varying characteristic deviation degree of each of the control factors, and determining a weight factor of each of the control factors based on a target time sequence of the contribution factor; Determining a reward and punishment coefficient based on the contribution factor and the weight factor of each of the control factors, and updating a benefit benchmark value of each execution action of the energy storage device based on the reward and punishment coefficient; the benefit benchmark value is determined based on a Q-learning algorithm; Determine the charging and discharging state of the energy storage device based on the updated benefit value.

[0005] Illustratively, the time-varying characteristic value is determined based on the historical sampling data sequence and the current sampling data of the control factor, including: calculating the average value of the historical sampling data in the historical sampling data sequence, and calculating the absolute value of the difference between the average value and the current sampling data, denoted as a first time-varying factor; in the space coordinate system in which the characteristic sphere is located, the variance of the slope of the line connecting the coordinate points of adjacent historical sampling data in the historical sampling data sequence is calculated, denoted as a second time-varying factor; the time-varying characteristic value is determined based on the first time-varying factor and the second time-varying factor.

[0006] Illustratively, the contribution factor of each control factor is determined based on the discrete degree of the sector height and the time-varying characteristic deviation degree of each control factor, including: calculating the variance of the time-varying characteristic value of each control factor, denoted as the discrete degree of the sector height; for each control factor, the absolute value of the difference between the time-varying characteristic value and the time-varying characteristic values of other control factors is calculated and averaged, and the obtained average value is subjected to target exponential operation to obtain the time-varying characteristic deviation degree; the contribution factor of the corresponding control factor is determined based on the discrete degree of the sector height and the time-varying characteristic deviation degree.

[0007] Illustratively, the weight factor of each control factor is determined based on the target time sequence of the contribution factor, including: calculating the average value of all the contribution factors of the control factor at different time points in the target time sequence, denoted as a first weight influence parameter; the target time sequence reflects the change rule of the contribution factor of the control factor over time; a second weight influence parameter is determined based on the sum of the difference between the contribution factor and the first weight influence parameter at different time points in the target time sequence; the weight factor is determined based on the first weight influence parameter and the second weight influence parameter.

[0008] Illustratively, the reward and punishment coefficient is determined based on the contribution factor and the weight factor of each control factor, including: processing the contribution factor of each control factor based on a target activation function, denoted as an error adjustment factor of each control factor; calculating the product of the error adjustment factor and the weight factor of the control factor, denoted as a reward and punishment influence degree of each control factor; the average value of the reward and punishment influence degree of each control factor is calculated to obtain the reward and punishment coefficient.

[0009] Illustratively, the benefit reference value of each execution action of the energy storage device is updated based on the reward and punishment coefficient, including: calculating the product of the reward and punishment coefficient and the benefit reference value of each execution action, denoted as the updated benefit value of each execution action.

[0010] Exemplarily, the execution actions include a charging action, a silent action and a discharging action, and the determining the charging and discharging strategy of the energy storage device based on the updated benefit value includes: setting the charging and discharging state of the energy storage device based on the execution action corresponding to the maximum updated benefit value.

[0011] Exemplarily, the control factors include energy storage device state data, power grid operation data, renewable energy supply data and market environment data.

[0012] Correspondingly, the application also provides an adaptive and efficient charging and discharging control device of an intelligent energy storage system, comprising: a data acquisition module configured to acquire multi-dimensional data; the multi-dimensional data includes a plurality of dimensions of control factors affecting the charging and discharging of the energy storage device; a data processing module configured to allocate a sector to each of the control factors in a pre-set characteristic sphere, determine a time-varying characteristic value based on a historical sampling data sequence and a current sampling data of the control factors, and determine a sector height corresponding to the sector based on the time-varying characteristic value and an initial radius of the characteristic sphere; The data processing module is further configured to determine a contribution factor of each of the control factors based on a discrete degree of the sector height and a time-varying characteristic deviation degree of each of the control factors, and determine a weight factor of each of the control factors based on a target time sequence of the contribution factor; The data processing module is further configured to determine a reward and punishment coefficient based on the contribution factor and the weight factor of each of the control factors, and update a benefit benchmark value of each execution action of the energy storage device based on the reward and punishment coefficient; the benefit benchmark value is determined based on a Q-learning algorithm; an energy storage control module configured to determine a charging and discharging state of the energy storage device based on an updated benefit value.

[0013] In addition, the application also provides an adaptive and efficient charging and discharging control system of an intelligent energy storage system, comprising an energy storage device, a sensor group, a communication module and an adaptive and efficient charging and discharging control device of an intelligent energy storage system; wherein: The sensor group is configured to acquire multi-dimensional data; The communication module is configured to transmit the multi-dimensional data acquired by the sensor group to the adaptive and efficient charging and discharging control device of the intelligent energy storage system; The adaptive and efficient charging and discharging control device of the intelligent energy storage system is configured to control the charging and discharging actions of the energy storage device through the adaptive and efficient charging and discharging control method of the intelligent energy storage system.

[0014] The application can have the following parts or all of the beneficial effects: In the adaptive and efficient charging and discharging control method of the intelligent energy storage system provided in the application, the multi-dimensional control factors are sectorally distributed by the characteristic sphere, and the height of each sector is determined based on the time-varying characteristic value of each control factor and the initial radius of the characteristic sphere, so that the change of data distribution can be perceived in real time, the problem that the Q learning algorithm cannot converge due to data distribution drift is solved, the influence of frequent adjustment of model parameters on stability is improved, and the adaptive adjustment of the charging and discharging state in the dynamic environment is ensured; by determining the contribution factor and the weight factor, the real-time influence degree of each control factor can be quantified, the control factor with greater influence degree is preferentially regulated, and the fineness of the charging and discharging control is improved; the reward and punishment coefficient is determined in combination with the weight factor and the contribution factor of each control factor, and the reward benchmark value based on Q learning is updated according to the reward and punishment coefficient, so that the charging and discharging control can cooperatively optimize multiple objectives, and the dynamic balance of each control factor in the charging and discharging control is realized. In summary, the application expands the scene adaptability of the traditional energy storage device charging and discharging control strategy, and improves the efficient operation capability of the energy storage system in multiple scenes such as high-proportion renewable energy power grid and power market.

[0015] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 A flowchart of the adaptive and efficient charging and discharging control method of the intelligent energy storage system according to an exemplary embodiment of the application is shown; Figure 2 A schematic diagram of the characteristic sphere in the adaptive and efficient charging and discharging control method of the intelligent energy storage system according to an exemplary embodiment of the application is shown; Figure 3 A schematic block diagram of the adaptive and efficient charging and discharging control device of the intelligent energy storage system according to an exemplary embodiment of the application is shown; Figure 4 A schematic block diagram of the adaptive and efficient charging and discharging control system of the intelligent energy storage system according to an exemplary embodiment of the application is shown. DETAILED DESCRIPTION

[0018] For further elaboration of the technical means and effects taken by the present application to achieve the predetermined object, the self-adaptive efficient charging and discharging control method, device and system of the intelligent energy storage system according to the present application, the specific implementation, structure, features and effects thereof are described in detail as follows in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0020] The self-adaptive efficient charging and discharging control method, device and system of the intelligent energy storage system provided by the present application are described in detail below in combination with the drawings.

[0021] Please refer to Figure 1 , which shows the method flowchart of the self-adaptive efficient charging and discharging control method of the intelligent energy storage system provided by one embodiment of the present application, as Figure 1 shown, the self-adaptive efficient charging and discharging control method of the intelligent energy storage system specifically includes the following steps: S110: collecting multi-dimensional data; the multi-dimensional data includes a plurality of dimensions of control factors affecting the charging and discharging of the energy storage device; S120: assigning a sector to each control factor in a pre-set characteristic sphere, determining a time-varying characteristic value based on the historical sampling data sequence and the current sampling data of the control factor, and determining the sector height based on the time-varying characteristic value and the initial radius of the characteristic sphere; S130: determining the contribution factor of each control factor based on the discrete degree of the sector height and the time-varying characteristic deviation degree of each control factor, and determining the weight factor of each control factor based on the target time sequence of the contribution factor; S140: determining the reward and punishment coefficient based on the contribution factor and the weight factor of each control factor, and updating the income benchmark value of each execution action of the energy storage device based on the reward and punishment coefficient; the income benchmark value is determined based on the Q-learning algorithm; S150: determining the charging and discharging state of the energy storage device based on the updated income value.

[0022] Next, each step of the self-adaptive efficient charging and discharging control method of the intelligent energy storage system described above is described in detail: In step S110, multi-dimensional data is collected; the multi-dimensional data includes a plurality of dimensions of control factors affecting the charging and discharging of the energy storage device.

[0023] In the embodiments of the present application, the multi-dimensional data refers to a set of various key parameters affecting the charging and discharging state control strategy of the energy storage device, covering multiple dimensions such as the state of the energy storage device itself, the operation state of the power grid, the output characteristics of renewable energy, and the market environment, and is used to comprehensively reflect the internal and external environment of the energy storage system operation.

[0024] In the embodiments of the present application, the control factor is a specific parameter of each dimension in the multi-dimensional data, which is a key variable directly or indirectly affecting the charging and discharging state control strategy of the energy storage device, and can be energy storage device state data, power grid operation data, renewable energy supply data, and market environment data.

[0025] For example, the energy storage device state data can include battery state and power conversion device state data. The battery state includes state of charge (SOC), state of health (SOH), temperature, and charging and discharging efficiency. Specifically, the state of charge is the percentage of the current battery remaining capacity (e.g., 60%); the state of health is the degree of battery capacity attenuation (e.g., the remaining capacity is 80% of the initial capacity); the temperature is the real-time temperature of the battery cell or module, which affects the charging and discharging efficiency and service life; and the charging and discharging efficiency is the proportion of energy loss in the energy conversion process (e.g., charging efficiency 92%, discharging efficiency 90%). The power conversion device state data can include power conversion system (PCS) operating parameters, including input and output voltage, current, power factor, and other data of the PCS.

[0026] The power grid operation data can include power grid load demand, power grid frequency, and voltage level data. The power grid load demand is the total power consumption in the current region (unit: kW / MW); the power grid frequency is the deviation of the actual frequency of the power grid from the standard frequency (e.g., 50 Hz ± 0.2 Hz), reflecting the stability of the power grid; and the voltage level is the voltage amplitude of the grid connection point (e.g., 400 V ± 10%).

[0027] The renewable energy supply data can include photovoltaic power generation power and wind power data. The photovoltaic power generation power is the actual output power of the current photovoltaic array (unit: kW); and the wind power is the real-time power generation of the wind turbine (unit: kW).

[0028] The market environment data can include price signal data, which is the time-of-use price of the electricity market, reflecting the electricity cost or benefit at different times.

[0029] Exemplarily, the collection of the multi-dimensional data can be implemented as follows: the state of charge is estimated by the battery management system (BMS) based on the coulomb counting method, the open circuit voltage method or the Kalman filter; the state of health is calculated by the BMS based on the cycle number, the internal resistance change and the capacity test; the real-time temperature of the battery cell or module is monitored by the temperature sensor in real time; the charge-discharge efficiency is calculated by the BMS based on the input-output energy ratio; the PCS operation parameters are collected by the built-in sensor of the PCS or the supervisory control and data acquisition (SCADA) system; the power grid load demand is obtained by the smart meter, the energy management system (EMS) or the application programming interface (API) of the power grid dispatching center; the power grid frequency is monitored by the frequency relay or the phasor measurement unit (PMU); the voltage level is collected by the voltage sensor or the smart meter; the photovoltaic power is obtained directly by the inverter monitoring data or calculated based on the irradiance data of the weather station; the wind power is obtained by the fan SCADA system or estimated based on the wind speed prediction model; and the real-time electricity price data is accessed by the power market interface or the energy management system.

[0030] It should be noted that the above scenarios are only exemplary, and the protection scope of the embodiments of the present application is not limited thereto. For example, the above multi-dimensional data can also include other dimensions of data related to the charge-discharge control strategy, and the above data collection method can also use other collection methods that can achieve the same function.

[0031] In step S120, a sector is assigned to each control factor in a pre-set characteristic sphere, a time-varying characteristic value is determined based on the historical sampling data sequence and the current sampling data of the control factor, and a corresponding sector height is determined based on the time-varying characteristic value and the initial radius of the characteristic sphere.

[0032] Due to the physical limitations of charge-discharge depth, rate and cycle life of the energy storage medium, excessive charge-discharge or frequent power fluctuations can accelerate material aging and even trigger thermal runaway risk, which needs to be avoided by control strategies to avoid exceeding the safety threshold; there are spatial and temporal differences in time-of-use electricity prices, renewable energy output fluctuations and grid load demand, and precise regulation of charge-discharge timing (such as low valley charging and high peak discharging) can reduce energy costs or generate revenue by participating in auxiliary services; as a flexible regulator of the power grid, energy storage needs to respond to dispatch instructions for frequency regulation, peak shaving or voltage support, and its charge-discharge power and capacity need to dynamically match the grid balancing demand to avoid exacerbating supply-demand imbalance due to disordered charging. Therefore, in order to ensure system safety, improve economic efficiency and optimize energy utilization efficiency, etc., the charging and discharging states of the energy storage system need to be controlled by considering the influence of the above multi-dimensional data, and since the key parameters such as energy storage device state data, grid operation data, renewable energy supply data and market environment data usually have significant time-varying characteristics (for example, electricity market prices may change structurally due to policy adjustments (such as redivision of peak and valley periods), and grid load patterns change with economic development and electricity usage habits), in order to dynamically adjust the charge-discharge power in a timely manner according to each data, the present embodiment innovatively introduces a feature sphere to dynamically describe multi-dimensional data.

[0033] In the present embodiment, the above-mentioned feature sphere refers to a standard sphere with a pre-set radius and a sphere center located at the origin of the spatial coordinate system, which is used to quantify the time-varying characteristics of multi-dimensional control factors through geometric variables, and is a geometric model for describing the dynamic changes of multi-dimensional data. For example, the initial radius of the feature sphere can be set to 1 as a reference parameter for calculating the height of the sector.

[0034] In the present embodiment, the above-mentioned sector is each equal division area of the feature sphere after being equally divided into N parts. Wherein, N is the number of dimensions of the control factor, and each sector corresponds to a control factor for visualizing the dynamic change of the control factor. For example, taking four control factors as an example, the above-mentioned allocation of sectors to each control factor in the pre-set feature sphere can be implemented as follows: the feature sphere is equally divided into four equal sectors, and each control factor is allocated a sector.

[0035] In the present embodiment, the above-mentioned historical sampling data sequence refers to a set of sampling data of a certain control factor at multiple time points in the past, which is used to calculate the average amplitude and trend of the control factor.

[0036] In the present embodiment, the above-mentioned current sampling data refers to the real-time sampling value of a certain control factor at the current time, reflecting the instantaneous state of the control factor.

[0037] In the embodiments of the present application, the time-varying characteristic value is a parameter for quantifying the dynamic change characteristic of the control factor at the current time, and comprehensively reflects the increasing / decreasing nature of the control factor and the deviation degree from the historical average value.

[0038] For example, the time-varying characteristic value based on the historical sampling data sequence and the current sampling data of the control factor can be determined as follows: the average value of the historical sampling data in the historical sampling data sequence is calculated, and the absolute value of the difference between the average value and the current sampling data is calculated, denoted as a first time-varying factor; in the space coordinate system in which the characteristic sphere is located, the variance of the slope of the line connecting the coordinate points of adjacent historical sampling data in the historical sampling data sequence is calculated, denoted as a second time-varying factor; and the time-varying characteristic value is determined based on the first time-varying factor and the second time-varying factor.

[0039] In a specific implementation manner of the embodiments of the present application, the time-varying characteristic value of the control factor can be determined as follows: an initial characteristic sphere with an initial radius of 1 is initialized, and is placed in a space coordinate system, with the sphere center coinciding with the origin of the coordinate system, as a reference geometric model for describing the dynamic change of the multi-dimensional control factor; the historical sampling data sequence and the current sampling data of all control factors are subjected to standardization of dispersion, so as to eliminate the dimensional difference of different dimensional data, and obtain normalized multi-dimensional data, to ensure that the control factors are comparable in the same scale; according to the number N of dimensions of the control factors, the characteristic sphere is equally divided into N sectors, and each sector uniquely corresponds to a control factor. For example, if there are two control factors, the characteristic sphere is divided into two semicircles; if there are four control factors, the characteristic sphere is divided into four equal sectors; for example, for the a-th dimensional control factor, the time-varying characteristic value of the a-th dimensional control factor at time t can be calculated by the following formula: wherein, is the time-varying characteristic value of the a-th dimensional control factor at time t; is the variance of the slope of adjacent data points in the historical sampling data sequence of the a-th dimensional control factor (i.e., the second time-varying factor), for reflecting the increasing / decreasing nature of the data of the control factor, and the greater the value of the slope variance , the more intense the fluctuation of the sampling data of the a-th dimensional control factor, and the greater the time-varying characteristic value of the control factor; is the average amplitude of the historical sampling data sequence of the a-th dimensional control factor, is the current sampling data, is the first time-varying factor, for reflecting the deviation degree of the current value of the a-th dimensional control factor from the historical average value, and the greater the value of the deviation degree , the greater the deviation degree of the current sampling data of the control factor from the historical average value, and the greater the time-varying characteristic value of the corresponding control factor.

[0040] In this embodiment, the height of the sector is determined by the product of the time-varying characteristic value of the control factor and the initial radius of the characteristic sphere, and is used to characterize the degree of influence of the control factor on the charging and discharging of the energy storage system at the current moment.

[0041] Specifically, after determining the time-varying characteristic values ​​of each control factor using the above method, assuming the initial radius of the characteristic sphere is 1, the determination of the corresponding sector height based on the time-varying characteristic values ​​and the initial radius of the characteristic sphere can be as follows: The time-varying characteristic value of each control factor at time t... Multiplying the value by the initial radius of the characteristic sphere yields the height of the sector corresponding to the control factor. At this point, the aforementioned characteristic sphere is no longer a standard sphere, such as... Figure 2 As shown, the initial feature sphere deforms due to the different heights of each sector. The degree of deformation directly reflects the difference in the dynamic influence of each control factor at the current moment. The larger the sector height, the more significant the immediate influence of the control factor.

[0042] In step S130, the contribution factor of each control factor is determined based on the dispersion of the sector height and the deviation of the time-varying characteristics of each control factor, and the weight factor of each control factor is determined based on the target time series of the contribution factor.

[0043] In this embodiment, the dispersion of the sector height is an index used to reflect the dispersion of the time-varying characteristic values ​​of each control factor at the current moment. For example, the dispersion of the sector height can be characterized by the variance of the time-varying characteristic values ​​of different control factors at that moment. The larger the variance of the time-varying characteristic values, the more significant the difference in dynamic changes of each control factor, and the lower the relative influence of a single control factor.

[0044] In this embodiment, the aforementioned deviation of the time-varying characteristics is used to quantify the difference in dynamic changes between the current control factor and other control factors. The greater the deviation of the time-varying characteristics, the greater the difference in dynamic changes between the current control factor and other control factors, proving that the time-varying characteristics of the current control factor are more unique and its influence on charge / discharge state control is more prominent.

[0045] In the embodiments of this application, the above-mentioned contribution factor is a parameter used to quantify the degree of influence of the control factor on the state of charge and discharge control at the current moment; the larger the value of the contribution factor, the more significant the influence of the control factor on the state of charge and discharge control at the current moment.

[0046] Exemplarily, the contribution factor of each control factor can be determined according to the dispersion degree of the sector height and the deviation degree of the time-varying characteristic of each control factor as follows: calculating the variance of the time-varying characteristic value of each control factor, denoted as the dispersion degree of the sector height; for each control factor, calculating the absolute value of the difference between the time-varying characteristic value of the current control factor and that of other control factors and averaging the values, and performing an exponential operation on the obtained average value to obtain the time-varying characteristic deviation degree; and determining the contribution factor of the corresponding control factor based on the dispersion degree of the sector height and the time-varying characteristic deviation degree.

[0047] Specifically, taking the a-th dimensional control factor as an example, the contribution factor of the a-th dimensional control factor at time t can be calculated by the following formula: wherein, is the contribution factor of the a-th dimensional control factor at time t; is the dispersion degree of the sector height, and the greater the dispersion degree of the sector height, the more significant the dynamic change difference of each control factor, that is, under the influence of multiple factors, the relative influence degree of the a-th dimensional control factor at time t is lower, and the contribution factor of the a-th dimensional control factor at time t is also smaller; is the time-varying characteristic value of the a-th dimensional control factor at time t, is the time-varying characteristic value of the b-th dimensional control factor at time t, is the absolute value of the average difference between the time-varying characteristic value of the a-th dimensional control factor and that of all other control factors (b=1, 2,..., N), reflecting the deviation degree of the a-th dimensional control factor relative to other dimensional control factors, wherein N is the number of dimensions of the control factors; is an exponential function with a natural constant as the base (i.e., the above exponential operation), used to amplify the time-varying characteristic difference between the current control factor and other dimensional control factors, so that it occupies a higher weight in the current time charging and discharging control, and realizes the priority response to the key changes; is the time-varying characteristic deviation degree, and the greater the value, the more unique the time-varying characteristic of the current control factor, the more prominent the influence on the charging and discharging state control, and the greater the corresponding contribution factor.

[0048] In the embodiments of the present application, the target time sequence is a sequence of contribution factors of a certain control factor at multiple continuous time points, reflecting the change rule of the contribution factors of the control factor over time, and is used to analyze the long-term influence trend of the control factor on the charging and discharging state control of the energy storage system. At this time, the control function of different dimensions of data is established by obtaining each three-dimensional model of the feature sphere at different time points and by analyzing the changes of multiple three-dimensional models in space. The control function can be used to quantify the mapping function of the influence of each dimension control factor on the charging and discharging state of the energy storage system. The continuous change of different three-dimensional models in the spatial position represents the importance of different dimension control factors at different time points. If the importance of a certain dimension control factor is continuously large, it reflects that the control factor has a greater influence on the charging and discharging of the energy storage system.

[0049] In the embodiments of the present application, the weight factor is a parameter for quantifying the overall importance of the control factor in the long-term time sequence, which can be calculated based on the target time sequence of the control factor. The greater the weight factor, the more critical the influence of the control factor on the charging and discharging state control in the long-term operation.

[0050] For example, the weight factor of each control factor based on the target time sequence of the contribution factor can be determined as follows: calculating the average value of all contribution factors of the control factor at different time points in the target time sequence, denoted as a first weight influence parameter; determining a second weight influence parameter based on the sum of the difference between the contribution factor and the first weight influence parameter at different time points in the target time sequence; and determining the weight factor based on the first weight influence parameter and the second weight influence parameter.

[0051] Specifically, taking the a-th dimension control factor as an example, the weight factor of the a-th dimension control factor can be calculated by the following formula: wherein, is the weight factor of the a-th dimension control factor; is the first weight influence parameter, reflecting the average contribution degree of the a-th dimension control factor in the target time sequence. The greater the value, the more critical the influence of the a-th dimension control factor on the charging and discharging state control in the long-term operation; is the contribution factor of the a-th dimension control factor at time t, is the average value of the contribution factors of all control factors at time t, and n is the number of time points included in the target time sequence (i.e., the number of three-dimensional models corresponding to the feature sphere in the target time sequence); The greater the value of ( ), the more significant the contribution factor of the a-th dimension control factor at time t is higher than other control factors, and the more significant the influence on the weight factor after long-term accumulation, is the cumulative value of the contribution difference of the a-th dimension control factor relative to other control factors in the long-term time sequence, is an exponential function with a natural constant as the base, used to amplify the relative prominence of the contribution factor of the a-th dimension control factor in the long-term operation. When the contribution factor of the a-th dimension control factor is continuously higher than the average level of other dimension control factors in the same period, the cumulative difference is positive and the greater the value, the greater the output value of the exponential function, thereby strengthening the long-term importance of the a-th dimension control factor. is the second weight influence parameter, and the greater the value, the greater the influence of the a-th dimension control factor on the charging and discharging control of the energy storage system in the long-term operation.

[0052] In step S140, the reward and punishment coefficient is determined based on the contribution factor and the weight factor of each control factor, and the reward and punishment coefficient is used to update the benefit reference value of each execution action of the energy storage device. The benefit reference value is determined based on the Q learning algorithm.

[0053] In the embodiments of the present application, the reward and punishment coefficient is a dynamic parameter determined by combining the real-time contribution degree (contribution factor) and the long-term importance (weight factor) of each control factor, and reflects the importance of each control factor in the process of charging and discharging control of the energy storage system at the current time. The greater the value, the more important the long-term benefit driven by the current key control factor in the charging and discharging control.

[0054] For example, the reward and punishment coefficient based on the contribution factor and the weight factor of each control factor can be determined as follows: the contribution factor of each control factor is processed based on a target activation function, and is recorded as the error adjustment factor of each control factor; the product of the error adjustment factor and the weight factor of the control factor is calculated, and is recorded as the reward and punishment influence degree of each control factor; and the average value of the reward and punishment influence degree of each control factor is calculated to obtain the reward and punishment coefficient.

[0055] Specifically, taking time t as an example, the reward and punishment coefficient can be calculated by the following formula: wherein, is the reward and punishment coefficient corresponding to time t; is the target activation function, used to dynamically adjust the influence weight of the contribution factor of the control factor on the reward and punishment coefficient, is the error adjustment factor of the a-th dimension control factor. Taking the a-th dimension control factor as an example, the specific formula for calculating the error adjustment factor of the a-th dimension control factor can be as follows: In the above formula, when the contribution factor of the a-th dimension control factor is significantly higher than other control factors, the output of the activation function tends to 1, which can significantly increase the weight of the control factor in the reward and punishment coefficient, prompting the energy storage system to pay more attention to the long-term benefits driven by the control factor (for example, preferentially discharging at the peak period of electricity price to obtain high benefits). When the contribution factor of a control factor is small, the output of the activation function is close to 0.5, which can reduce the influence of the control factor on the reward and punishment coefficient, avoid the error caused by the over-reliance of the energy storage system on the long-term benefit estimation in the charging and discharging control, and thus maintain the stability of the long-term strategy. is the weight factor of the a-th dimension control factor, reflecting the overall influence of the control factor on the charging and discharging control in the long term, and the greater the value, the more critical the long-term influence is; is the reward and punishment influence degree of the a-th dimension control factor, and the value realizes the fusion of long-term trend and short-term fluctuation, and the greater the value, the more the influence of the control factor needs to be focused on in the charging and discharging control of the energy storage system; is the dimension number of the control factor, and the average value ensures that the synergistic effect of the multi-dimensional control factor is taken into account, avoiding the dominance of a single factor in decision-making.

[0056] In the embodiments of the present application, the above-mentioned benefit benchmark value is a core parameter obtained based on a Q-learning algorithm for evaluating the long-term cumulative reward of a "state-action" combination, which is used to represent the expected long-term benefit of a certain charging and discharging action (such as charging, discharging, and silence) in the current state. The Q-learning algorithm is a model-free reinforcement learning algorithm, and its core idea is to learn the optimal charging and discharging control strategy through the continuous interaction between the agent (energy storage system control algorithm) and the dynamic operating environment (such as power grid demand, electricity price fluctuation, battery state, and renewable energy output) to maximize the long-term cumulative reward. It should be noted that the implementation of determining the benefit benchmark value by the Q-learning algorithm is the same as the related technology, and will not be described here.

[0057] Exemplarily, the above-mentioned updating of the benefit benchmark value of each execution action of the energy storage device based on the reward and punishment coefficient can be implemented as follows: calculating the product of the reward and punishment coefficient and the benefit benchmark value of each execution action, denoted as the updated benefit value of each execution action. The above-mentioned execution actions include charging action, silence action and discharging action. Specifically, taking time t as an example, the above-mentioned updated benefit value can be calculated by the following formula: wherein, is the updated benefit value at time t; is the reward and punishment coefficient corresponding to time t; is the benefit benchmark value obtained based on the Q-learning algorithm.

[0058] In step S150, the charging and discharging state of the energy storage device is determined based on the updated benefit value.

[0059] In the embodiments of the present application, the updated benefit value is used to quantify the long-term expected benefit of a certain charging and discharging action (such as charging, discharging, or silence) under the current system state; the greater the value, the better the long-term cumulative reward of the action under the current working condition.

[0060] In the embodiments of the present application, the charging and discharging state of the energy storage device refers to the operation and power parameters of the energy storage system at a specific time, including three core states of charging (including charging power), discharging (including discharging power), and silence (neither charging nor discharging), which need to be dynamically adjusted according to the updated benefit value to adapt to changes in the environment such as grid demand and battery state.

[0061] For example, if it is found through evaluation based on the updated benefit value that there is excess photovoltaic output during the noon period and the electricity price is at a low point, the benefit value of the charging action is the largest, at which time the energy storage device is set to the charging state to preferentially store excess photovoltaic power; while in the evening period, when the grid load is detected to be at a peak and the electricity price is high, the benefit value of the discharging action is the largest, so the energy storage device is set to the discharging state to support the load by discharging to the grid and obtain the price difference benefit.

[0062] In addition, the embodiments of the present application can also include functions of multi-time scale rolling optimization, dynamic correction of safety boundaries, and instruction execution and closed-loop feedback. Among them, the multi-time scale rolling optimization function formulates a charging and discharging state plan (such as charging power during the low valley period) at the hour level resolution in the day-ahead planning layer, participates in the power market bidding, responds to the grid frequency deviation and power fluctuation at the minute / second level in the real-time control layer, dynamically adjusts the charging and discharging power (such as increasing the discharging power when the load suddenly increases), and corrects the charging and discharging control strategy through model predictive control to offset the prediction error; the dynamic correction of safety boundaries function updates the constraint conditions of the charging and discharging state (such as reducing the upper limit of SOC from 90% to 85% when SOH decreases) according to the battery health state decay and environmental temperature changes, avoids excessive charging and discharging, and prolongs the battery life; the instruction execution and closed-loop feedback function sends the optimized charging and discharging state instruction (such as discharging 300 kW) to the power conversion system, synchronously monitors the execution effect, and takes the actual charging and discharging efficiency, SOC change, and benefit increment as feedback signals to continuously iterate and optimize the charging and discharging state.

[0063] The above describes the scheme provided by the embodiments of the present application from the method aspect. To implement the above functions, the hardware structure and / or software module corresponding to the execution of each function are included. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in the form of hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0064] Correspondingly, the embodiments of the present application provide an adaptive and efficient charging and discharging control device of an intelligent energy storage system, as shown in Figure 3 The adaptive and efficient charging and discharging control device 300 of the intelligent energy storage system can include a data acquisition module 310, a data processing module 320, and an energy storage control module 330, wherein: The data acquisition module is configured to acquire multi-dimensional data. The multi-dimensional data includes a plurality of control factors affecting the charging and discharging of the energy storage device. The data processing module is configured to assign a sector to each control factor in a pre-set characteristic sphere, determine a time-varying characteristic value based on a historical sampling data sequence and a current sampling data of the control factor, and determine a sector height corresponding to the time-varying characteristic value and an initial radius of the characteristic sphere. The data processing module is further configured to determine a contribution factor of each control factor based on a discrete degree of the sector height and a time-varying characteristic deviation degree of each control factor, and determine a weight factor of each control factor based on a target time sequence of the contribution factor. The data processing module is further configured to determine a reward and punishment coefficient based on the contribution factor and the weight factor of each control factor, and update a benefit benchmark value of each execution action of the energy storage device based on the reward and punishment coefficient. The benefit benchmark value is determined based on a Q-learning algorithm. The energy storage control module is configured to determine a charging and discharging state of the energy storage device based on the updated benefit value.

[0065] The specific implementation details of the adaptive and efficient charging and discharging control device of the intelligent energy storage system have been described in detail in the corresponding position of the adaptive and efficient charging and discharging control method of the intelligent energy storage system, and therefore will not be described here.

[0066] In addition, the embodiments of the present application also provide an adaptive and efficient charging and discharging control system of an intelligent energy storage system, as shown in Figure 4 The adaptive and efficient charging and discharging control system 400 of the intelligent energy storage system can include a sensor group 410, a communication module 420, and an adaptive and efficient charging and discharging control device 300 of an intelligent energy storage system, wherein: a sensor group for collecting multi-dimensional data; a communication module for transmitting the multi-dimensional data collected by the sensor group to an adaptive high-efficiency charging and discharging control device of the intelligent energy storage system; the adaptive high-efficiency charging and discharging control device of the intelligent energy storage system is configured to control the charging and discharging actions of the energy storage device by executing the adaptive high-efficiency charging and discharging control method of the intelligent energy storage system.

[0067] The specific implementation details of the adaptive high-efficiency charging and discharging control system of the intelligent energy storage system have been described in detail in the corresponding positions of the adaptive high-efficiency charging and discharging control method of the intelligent energy storage system, and therefore will not be described here.

[0068] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0069] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. An adaptive efficient charging and discharging control method for a smart energy storage system, characterized in that, The method comprises: Collecting multi-dimensional data; the multi-dimensional data comprises a plurality of dimensions of control factors affecting the charging and discharging of the energy storage device; Assigning a sector to each of the control factors in a pre-set characteristic sphere, determining a time-varying characteristic value based on a historical sampling data sequence and current sampling data of the control factor, and determining a sector height corresponding to the control factor based on the time-varying characteristic value and an initial radius of the characteristic sphere; Determining a contribution factor of each of the control factors based on a discrete degree of the sector height and a time-varying characteristic deviation degree of each of the control factors, and determining a weight factor of each of the control factors based on a target time sequence of the contribution factor; Determining a reward and punishment coefficient based on the contribution factor and the weight factor of each of the control factors, and updating a benefit benchmark value of each of the actions performed by the energy storage device based on the reward and punishment coefficient; the benefit benchmark value is determined based on a Q learning algorithm; Determining a charging and discharging state of the energy storage device based on the updated benefit value.

2. The adaptive efficient charging and discharging control method of the intelligent energy storage system according to claim 1, characterized in that, The method comprises: Calculating an average value of historical sampling data in the historical sampling data sequence, and calculating an absolute value of a difference between the average value and the current sampling data, denoted as a first time-varying factor; In a space coordinate system in which the characteristic sphere is located, calculating a variance of a slope of a line connecting coordinate points of adjacent historical sampling data in the historical sampling data sequence, denoted as a second time-varying factor; Determining the time-varying characteristic value based on the first time-varying factor and the second time-varying factor. 3.The adaptive efficient charge-discharge control method of the intelligent energy storage system according to claim 1, characterized in that, The method comprises: Calculating a variance of the time-varying characteristic value of each of the control factors, denoted as a discrete degree of the sector height; For each of the control factors, calculating an average value of absolute values of differences between the time-varying characteristic value of the control factor and time-varying characteristic values of other control factors, and performing target exponential operation on the obtained average value to obtain the time-varying characteristic deviation degree; Determining the contribution factor of the corresponding control factor based on the discrete degree of the sector height and the time-varying characteristic deviation degree. 4.The adaptive efficient charge-discharge control method of the intelligent energy storage system according to claim 3, characterized in that, The method comprises: Calculating an average value of all the contribution factors of the control factor at different time points in the target time sequence, denoted as a first weight influence parameter; the target time sequence reflects a change rule of the contribution factor of the control factor over time; Determining a second weight influence parameter based on a sum of differences between the contribution factor and the first weight influence parameter at different time points in the target time sequence; Determining the weight factor based on the first weight influence parameter and the second weight influence parameter.

5. The adaptive efficient charging and discharging control method of the intelligent energy storage system according to claim 1, characterized in that, The method comprises: Processing the contribution factor of each of the control factors based on a target activation function, denoted as an error adjustment factor of each of the control factors; Calculating a product of the error adjustment factor of the control factor and the weight factor, denoted as a reward and punishment influence degree of each of the control factors; Calculate the average of the reward and punishment influence degree of each control factor to obtain the reward and punishment coefficient. 6.The adaptive efficient charge-discharge control method of the intelligent energy storage system according to claim 1, characterized in that, The reward and punishment coefficient is used to update the benefit reference value of each execution action of the energy storage device. The product of the reward and punishment coefficient and the benefit reference value of each execution action is calculated, which is recorded as the updated benefit value of each execution action.

7. The adaptive efficient charging and discharging control method of the intelligent energy storage system according to claim 1, characterized in that, The execution action includes charging action, silent action and discharging action, and the charging and discharging strategy of the energy storage device is determined based on the updated benefit value. The charging and discharging state of the energy storage device is set based on the execution action corresponding to the maximum updated benefit value. 8.The adaptive efficient charge-discharge control method of the intelligent energy storage system according to claim 1, wherein, The control factor includes energy storage device state data, power grid operation data, renewable energy supply data and market environment data.

9. An adaptive high efficiency charge-discharge control device of a smart energy storage system, characterized in that, It includes: A data acquisition module is used to acquire multi-dimensional data; The multi-dimensional data includes a plurality of dimensions of control factors affecting the charging and discharging of the energy storage device; The data processing module is used to assign a sector to each control factor in a pre-set characteristic sphere, determine a time-varying characteristic value based on the historical sampling data sequence and the current sampling data of the control factor, and determine the sector height based on the time-varying characteristic value and the initial radius of the characteristic sphere; The data processing module is also used to determine the contribution factor of each control factor based on the dispersion degree of the sector height and the time-varying characteristic deviation degree of each control factor, and determine the weight factor of each control factor based on the target time sequence of the contribution factor; The data processing module is also used to determine the reward and punishment coefficient based on the contribution factor and the weight factor of each control factor, and update the benefit reference value of each execution action of the energy storage device based on the reward and punishment coefficient; The benefit reference value is determined based on the Q learning algorithm; The energy storage control module is used to determine the charging and discharging state of the energy storage device based on the updated benefit value.

10. An adaptive high efficiency charge-discharge control system for a smart energy storage system, characterized in that, It includes an energy storage device, a sensor group, a communication module and an adaptive efficient charging and discharging control device of an intelligent energy storage system according to claim 9; wherein: The sensor group is used to acquire multi-dimensional data; The communication module is used to transmit the multi-dimensional data collected by the sensor group to the adaptive efficient charging and discharging control device of the intelligent energy storage system; The adaptive efficient charging and discharging control device of the intelligent energy storage system is used to control the charging and discharging action of the energy storage device by executing the adaptive efficient charging and discharging control method of the intelligent energy storage system according to any one of claims 1-8.

Images