Supercomputing center battery charging and discharging strategy optimization method and system based on reinforcement learning

Abstract

The present invention provides a method and system for optimizing the charging and discharging strategies of batteries in a supercomputing center based on reinforcement learning. The method involves acquiring a real-time operating data set of a supercomputing center battery system, extracting load features from the real-time operating data set, generating a load feature set for the battery system, and inputting the load feature set into a pre-trained reinforcement learning strategy network. The load feature set is then subjected to charge and discharge strategy priority matching, generating a priority ranking result for a target charge and discharge strategy, adjusting the current charge and discharge strategy set based on the priority ranking result, generating an optimized charge and discharge strategy set, executing charge and discharge operations based on the optimized charge and discharge strategy set, and collecting battery performance feedback data after the charge and discharge operations to iteratively update the strategy matching parameters of the reinforcement learning strategy network. The present invention overcomes the problems of overcharging and discharging and power oscillation caused by the delayed response of environmental parameters in traditional fixed strategies, and achieves autonomous decision-making for multi-objective optimization without relying on artificial empirical rules.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and in particular to a method and system for optimizing battery charging and discharging strategies in a supercomputing center based on reinforcement learning. Background Art

[0002] With the exponential growth in computing power demand at supercomputing centers, optimizing the battery charging and discharging strategies of their power supply systems has become a critical step in ensuring stable system operation. Existing technologies typically employ fixed-threshold charge and discharge trigger mechanisms, controlling battery charging and discharging behavior through preset current and voltage thresholds, or selecting strategies based on load fluctuations. However, supercomputing center battery systems are often subject to the coupling of loads and complex environments. There is a strong nonlinear correlation between current transient shocks, voltage fluctuation patterns, and temperature response characteristics. Traditional methods, unable to analyze multi-parameter coupling effects in real time, lead to delayed strategy generation, frequently causing overcharging and discharging or power allocation mismatch issues. The predefined rules of fixed strategy libraries are difficult to adapt to the evolving characteristics of load fluctuations, and after strategy execution, there is a lack of feedback and correction capabilities for battery performance degradation paths, resulting in increased battery cycle life loss. Furthermore, existing solutions rigidly bind charge and discharge trigger conditions to power parameters, limiting strategy flexibility. In response to sudden load increases or sudden changes in environmental parameters, system protection mechanisms are easily triggered, causing power outages, seriously impacting the continuity of supercomputing tasks. Summary of the Invention

[0003] The present invention provides a method and system for optimizing the battery charging and discharging strategy of a supercomputing center based on reinforcement learning.

[0004] In a first aspect, an embodiment of the present invention provides a method for optimizing a battery charging and discharging strategy in a supercomputing center based on reinforcement learning, comprising:

[0005] Acquire real-time operating data sets of the supercomputing center battery system, wherein the real-time operating data sets include battery status parameter sets, load fluctuation parameter sets, and environmental monitoring parameter sets;

[0006] Performing load feature extraction processing on the real-time operation data set to generate a load feature set of the battery system; the load feature set includes current change trend features, voltage fluctuation pattern features, and temperature correlation response features;

[0007] Inputting the load feature set into a pre-trained reinforcement learning strategy network, performing charge and discharge strategy priority matching processing on the load feature set through the reinforcement learning strategy network, and generating a priority ranking result of a target charge and discharge strategy;

[0008] Adjusting the current charge and discharge strategy set according to the priority sorting result to generate an optimized charge and discharge strategy set; each charge and discharge strategy in the optimized charge and discharge strategy set includes a charge and discharge trigger condition parameter and a power allocation parameter;

[0009] A charge and discharge operation is performed based on the optimized charge and discharge strategy set, and battery performance feedback data after the charge and discharge operation is collected to iteratively update the strategy matching parameters of the reinforcement learning strategy network.

[0010] In a second aspect, an embodiment of the present invention provides a computer system, including:

[0011] a memory storing a computer program;

[0012] A processor is used to load the computer program to implement the supercomputing center battery charging and discharging strategy optimization method based on reinforcement learning as described above.

[0013] The reinforcement learning-based supercomputing center battery charging and discharging strategy optimization method provided by the present invention generates a load feature set by real-time integration of battery state parameters, load fluctuation parameters and environmental monitoring parameters, comprehensively capturing the current change law, voltage fluctuation characteristics and temperature response mode during battery operation, so that the reinforcement learning strategy network can deeply analyze the electric-thermal-load coupling relationship and generate strategy priority ranking; optimize the trigger condition parameters and power allocation parameters of the decoupling design in the charging and discharging strategy set, realize the coordinated optimization of charging and discharging timing and execution intensity, and effectively balance the load demand matching accuracy and battery performance protection strength; drive the iterative update of strategy network parameters by real-time collection of performance feedback data after strategy execution, establish an effective adaptation mechanism for strategy generation and battery state changes, significantly improve the generalization ability of charging and discharging strategies for complex load scenarios in supercomputing centers, and improve the system power supply stability while reducing the risk of battery performance degradation, overcome the overcharging and power oscillation problems caused by the delayed response of environmental parameters in traditional fixed strategies, and achieve autonomous decision-making for multi-objective optimization without relying on artificial experience rules. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 This is a flowchart of a method for optimizing battery charging and discharging strategies in a supercomputing center based on reinforcement learning provided by an embodiment of the present invention.

[0016] Figure 2 It is a schematic diagram of the composition of a computer system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0018] See also Figure 1 , Figure 1 A flowchart of a method for optimizing a battery charging and discharging strategy in a supercomputing center based on reinforcement learning is provided in an embodiment of the present invention. The method for optimizing a battery charging and discharging strategy in a supercomputing center based on reinforcement learning can be executed by a computer system. The method for optimizing a battery charging and discharging strategy in a supercomputing center based on reinforcement learning may include the following steps:

[0019] Step S100: acquiring a real-time operating data set of the supercomputing center battery system in real time, where the real-time operating data set includes a battery state parameter set, a load fluctuation parameter set, and an environmental monitoring parameter set.

[0020] The real-time operating data set refers to a series of data collected in real time during the operation of the supercomputing center's battery system, reflecting the system's operating status. The battery status parameter set describes the battery's own state, covering parameters such as battery voltage, current, temperature, and state of charge. These parameters can reflect the battery's current performance and health. The load fluctuation parameter set contains data on the changes in the supercomputing center's load, such as load power changes and current fluctuations, reflecting the dynamic changes in the supercomputing center's workload at different times. The environmental monitoring parameter set is data obtained by monitoring the battery system's environment, including parameters such as ambient temperature, humidity, and air pressure. Environmental factors can have a significant impact on battery performance and lifespan.

[0021] In order to obtain this data in real time, a variety of sensors and monitoring devices can be used. For the battery status parameter set, a voltage sensor can be used to measure the battery voltage in real time, a current sensor can collect the battery current, a temperature sensor can monitor the battery temperature, and a state of charge monitoring device can obtain the battery state of charge. For the load fluctuation parameter set, a power sensor can be used to measure the load power of the supercomputing center, and a current transformer can be used to collect changes in load current. For the environmental monitoring parameter set, a temperature sensor can be used to measure the ambient temperature, a humidity sensor can be used to obtain the ambient humidity, and a pressure sensor can be used to monitor the ambient air pressure. For example, in a supercomputing center, corresponding sensors can be installed at key locations of the battery system to collect data at regular intervals and transmit the collected data to the data processing center for storage and analysis.

[0022] Step S200: performing load feature extraction processing on the real-time operation data set to generate a load feature set of the battery system; the load feature set includes current change trend features, voltage fluctuation pattern features, and temperature correlation response features.

[0023] Load feature extraction and processing analyzes and processes real-time operational data sets to extract key features that reflect the load characteristics of the battery system. A load feature set is a set of characteristics used to describe the load characteristics of the battery system. The current change trend feature reflects the temporal variation of the battery current, the voltage fluctuation pattern feature reflects the fluctuation pattern of the battery voltage, and the temperature correlation response feature indicates the correlation between temperature changes and load fluctuations.

[0024] As an embodiment, step S200 performs load feature extraction processing on the real-time operation data set to generate a load feature set of the battery system, which may specifically include the following steps S210 to S240:

[0025] Step S210: performing time domain trend analysis on the current parameter sequence in the battery state parameter set, extracting the fluctuation period characteristics and peak interval distribution characteristics of the current parameter sequence, and linearly superimposing the fluctuation period characteristics and peak interval distribution characteristics to generate current change trend characteristics.

[0026] Time-domain trend analysis analyzes current parameter sequences in the time domain to identify their changing patterns and trends. Fluctuation periodicity refers to the periodic nature of fluctuations in the current parameter sequence, reflecting the periodic nature of current fluctuations. Peak interval distribution refers to the distribution of time intervals between adjacent peaks in the current parameter sequence, demonstrating the regularity of current peak occurrence. Linear superposition combines the fluctuation periodicity and peak interval distribution features according to a predefined linear relationship to produce a comprehensive current trend profile.

[0027] As an embodiment, step S210 performs a time domain trend analysis on the current parameter sequence in the battery state parameter set to extract the fluctuation period characteristics and peak interval distribution characteristics of the current parameter sequence. Specifically, the following steps S211 to S215 may be included:

[0028] Step S211: performing sliding window segmentation processing on the current parameter sequence to generate multiple current parameter subsequences.

[0029] Sliding window segmentation is the process of segmenting the current parameter sequence according to the set window size and sliding step size to obtain multiple subsequences. The window size refers to the number of current data points contained in each subsequence, and the sliding step size refers to the distance the window moves each time. Through the sliding window segmentation process, a long current parameter sequence can be divided into multiple shorter subsequences, facilitating subsequent analysis and processing. For example, if the current parameter sequence has 1000 data points, the window size is set to 100, and the sliding step size is set to 10, then (1000-100) / 10+1=91 current parameter subsequences can be obtained.

[0030] Step S212: performing local extreme value detection on each current parameter subsequence, and extracting peak point positions and time interval data between adjacent peak points.

[0031] Local extreme value detection locates the local maximum and minimum points within each current parameter subsequence. A peak point is a point where the current value reaches its maximum value within a local range. Local extreme value detection can determine the location of peak points within a current parameter subsequence and calculate the time interval between adjacent peak points. In practice, a differential method can be used to calculate the difference between adjacent data points. When the difference changes from positive to negative, the corresponding point is identified as a peak point. For example, for a current parameter subsequence [1, 3, 5, 4, 2, 6, 5, 3], the differential method can detect that the peak points are 5 and 6, and the time interval between adjacent peak points is three data points.

[0032] Step S213: Count the distribution histograms of all time interval data, and calculate the kurtosis coefficient and skewness coefficient of the distribution histograms.

[0033] A distribution histogram is used to display the distribution of data. The time interval data for all adjacent peak points are grouped, the frequency of the data within each group is counted, and then a histogram is plotted. The kurtosis coefficient is a statistic that describes the peaked state of the data distribution and reflects the steepness of the data distribution. The skewness coefficient is a statistic that describes the skewness of the data distribution and reflects the degree of asymmetry of the data distribution. The kurtosis coefficient and skewness coefficient can be calculated using relevant functions in statistical software or programming languages. For example, in Python, the kurtosis and skew functions in the scipy.stats library can be used to calculate the kurtosis coefficient and skewness coefficient.

[0034] Step S214: generating peak interval distribution characteristics according to the kurtosis coefficient and the skewness coefficient.

[0035] The peak interval distribution feature is a feature vector constructed based on the kurtosis coefficient and skewness coefficient. It describes the distribution characteristics of the current peak interval. The kurtosis coefficient and skewness coefficient can be used as two components of the feature vector to form the peak interval distribution feature. For example, if the kurtosis coefficient is 2.5 and the skewness coefficient is 0.8, the peak interval distribution feature can be expressed as [2.5, 0.8].

[0036] Step S215: Perform Fourier transform processing on the current parameter sequence, extract the main frequency components and their corresponding amplitudes, and generate the fluctuation period characteristics according to the inverse of the main frequency components.

[0037] The Fourier transform process converts the current parameter sequence from the time domain to the frequency domain in order to analyze its frequency components. The primary frequency component refers to the frequency component with the largest amplitude in the frequency domain, corresponding to the main period of current fluctuation. The Fourier transform can be used to obtain the spectrum of the current parameter sequence, from which the primary frequency components and their corresponding amplitudes can be extracted. The fluctuation period characteristic is calculated based on the inverse of the primary frequency component, as frequency and period are reciprocals of each other. For example, if the primary frequency component is 5 Hz, the fluctuation period characteristic is 1 / 5 = 0.2 seconds.

[0038] Step S220: Perform frequency domain conversion processing on the voltage parameter sequence in the voltage fluctuation parameter set, extract the energy ratio characteristics of the voltage fluctuation component whose voltage is lower than the first preset frequency and the noise component whose voltage is greater than the second preset frequency, and calculate the correlation between the energy ratio characteristics and the mean variance characteristics of the voltage parameter sequence to generate the voltage fluctuation pattern characteristics.

[0039] Frequency domain conversion involves converting the voltage parameter sequence from the time domain to the frequency domain, for example using a Fourier transform. The first and second preset frequencies are pre-set frequency thresholds used to delineate between voltage fluctuation components and noise components (e.g., frequency band demarcation criteria based on battery characteristics). The voltage fluctuation component refers to the portion of the voltage below the first preset frequency, reflecting the low-frequency fluctuation characteristics of the voltage. The noise component refers to the portion of the voltage above the second preset frequency, reflecting the high-frequency noise characteristics of the voltage. The energy ratio feature is the ratio of the energy of the voltage fluctuation component to the energy of the noise component, reflecting the relative magnitude of the voltage fluctuation and noise. The mean-variance feature is the mean and variance of the voltage parameter sequence, describing the average voltage level and degree of fluctuation, respectively. Correlation calculation involves correlating the energy ratio feature with the mean-variance feature to generate a comprehensive voltage fluctuation pattern feature.

[0040] In specific implementation, the voltage parameter sequence is first subjected to a Fourier transform to obtain its spectrum. Then, according to the first preset frequency and the second preset frequency, the spectrum is divided into a voltage fluctuation component and a noise component, and their energies are calculated. Next, the mean and variance of the voltage parameter sequence are calculated. Finally, the correlation coefficient and other methods are used to calculate the correlation between the energy ratio feature and the mean variance feature to obtain the voltage fluctuation pattern feature. For example, if the first preset frequency is 10Hz and the second preset frequency is 100Hz, the energy of the voltage fluctuation component obtained by Fourier transform is 50J, the energy of the noise component is 10J, the mean of the voltage parameter sequence is 220V, and the variance is 100, then the energy ratio feature is 50 / 10=5. If the correlation coefficient is calculated to obtain a correlation degree of 0.8, then the voltage fluctuation pattern feature can be expressed as a comprehensive feature vector.

[0041] Step S230: constructing a response delay function of temperature change to load fluctuation based on the temporal correlation between the temperature parameter sequence in the environmental monitoring parameter set and the load fluctuation parameter set, and generating a temperature-related response feature by integrating the response delay function.

[0042] Timing correlation refers to the temporal relationship between a temperature parameter sequence and a set of load fluctuation parameters, reflecting the synchronization and delay between temperature changes and load fluctuations. The response delay function describes the response delay characteristics of temperature changes to load fluctuations. It indicates how long it takes for the temperature to change accordingly after a load fluctuation occurs. The integral operation integrates the response delay function over a set time interval to obtain the cumulative response effect of temperature changes to load fluctuations. The temperature-correlated response feature is constructed based on the results of the integral operation and is used to describe the degree of correlation between temperature changes and load fluctuations.

[0043] As an embodiment, step S230 constructs a response delay function of temperature change to load fluctuation based on the temporal correlation between the temperature parameter sequence in the environmental monitoring parameter set and the load fluctuation parameter set, and generates a temperature-related response feature by integrating the response delay function. Specifically, steps S231 to S235 may be included:

[0044] Step S231: performing timestamp alignment processing on the temperature data points in the temperature parameter sequence and the load data points in the load fluctuation parameter set to generate a temperature-load time series pair set within the synchronization time window.

[0045] Timestamp alignment matches and aligns the data points in the temperature parameter sequence and the load fluctuation parameter set according to their timestamps, ensuring their temporal synchronization. The synchronization time window refers to a specific time interval within which temperature and load data are processed. A temperature-load time series pair set is a set of aligned temperature and load data points, each pair containing a temperature value and a corresponding load value. For example, if the temperature parameter sequence and the load fluctuation parameter set are [(t1, T1), (t2, T2), ...] and [(t1', L1), (t2', L2), ...], respectively, timestamp alignment pairs data points with similar timestamps, resulting in a set of temperature-load time series pairs within the synchronization time window: [(t1, T1, L1), (t2, T2, L2), ...].

[0046] Step S232: Determine the delay time corresponding to the maximum cross-correlation value of the temperature change relative to the load fluctuation based on the cross-correlation analysis of the temperature change rate and the load change rate in the temperature-load time series pair set.

[0047] Cross-correlation analysis is a method used to analyze the correlation between two time series. It determines the delay relationship between them by calculating their cross-correlation function. The temperature change rate refers to the change in temperature per unit time, and the load change rate refers to the change in load per unit time. The delay time corresponding to the maximum cross-correlation value refers to the time delay corresponding to the maximum value of the cross-correlation function. This delay time represents the response delay of temperature changes relative to load fluctuations. In specific implementation, the numpy.correlate function in Python can be used to calculate the cross-correlation function of the temperature change rate and the load change rate, then find the index corresponding to the maximum value of the cross-correlation function and convert this index into a time delay.

[0048] Step S233: Based on the delay time and the proportional relationship between the temperature and load change amplitudes in the temperature-load timing pair set, a piecewise linear response delay function of the temperature change to the load fluctuation is constructed; the piecewise linear response delay function includes temperature response slope parameters and delay compensation parameters corresponding to different load intervals.

[0049] A piecewise linear response delay function is a piecewise-defined linear function used to describe the response of temperature changes to load fluctuations. The temperature response slope parameter corresponding to different load intervals represents the rate at which temperature changes with load within different load ranges. The delay compensation parameter is determined based on the delay time and is used to compensate for the delay of temperature changes relative to load fluctuations. To construct the function, the load interval is first divided into multiple subintervals based on the load size. Then, within each subinterval, the temperature response slope parameter is calculated based on the proportional relationship between the temperature and load change amplitudes in the set of temperature-load time series pairs. Finally, the delay compensation parameter is determined based on the delay time to construct the piecewise linear response delay function. For example, if the load interval is divided into three subintervals [0, 100], [100, 200], and [200, 300], and the temperature response slope parameters k1, k2, and k3 are calculated for each subinterval, and the delay compensation parameter is d, then the piecewise linear response delay function can be represented as a piecewise function.

[0050] Step S234: Integrate the cumulative effect of the piecewise linear response delay function within the synchronization time window to generate a cumulative response energy distribution feature of the temperature change to the load fluctuation.

[0051] Integration processing integrates the piecewise linear response delay function within the synchronization time window to calculate the cumulative response effect of temperature changes on load fluctuations. The cumulative response energy distribution feature is a feature constructed based on the integration result, which is used to describe the temporal distribution of the cumulative response of temperature changes to load fluctuations. In specific implementation, numerical integration methods such as the trapezoidal integration method or the Simpson integration method can be used to integrate the piecewise linear response delay function. For example, using the trapezoidal integration method, the synchronization time window is divided into multiple small intervals. In each small interval, the piecewise linear response delay function is approximated as a linear function. The integral value of each small interval is then calculated. Finally, the integral values ​​of all small intervals are added together to obtain the cumulative response energy distribution feature.

[0052] Step S235: performing coupling calculation on the cumulative response energy distribution feature and the mean offset of the temperature parameter sequence, and outputting the temperature-related response feature.

[0053] The mean shift is the difference between the mean of the temperature parameter sequence and a reference value, reflecting the overall temperature deviation. The coupled calculation combines the cumulative response energy distribution characteristics and the mean shift to generate a comprehensive temperature-dependent response characteristic. In practice, the cumulative response energy distribution characteristics and the mean shift can be coupled using methods such as weighted summation. For example, if the cumulative response energy distribution characteristic is E, the mean shift is ΔT, and the weights are w1 and w2, respectively, the temperature-dependent response characteristic can be expressed as w1 × E + w2 × ΔT.

[0054] Step S240: performing multi-dimensional normalization fusion processing on the current change trend characteristics, the voltage fluctuation pattern characteristics and the temperature correlation response characteristics, and outputting a load feature set.

[0055] Multidimensional normalization and fusion normalizes feature data from different dimensions to the same dimension and range. These features are then fused to form a comprehensive load feature set. Normalization can be performed using a variety of methods, such as min-max normalization and Z-score normalization. Fusion can also be performed using methods such as vector concatenation and weighted summation. For example, if the current trend features are [1, 2, 3], the voltage fluctuation pattern features are [4, 5, 6], and the temperature-related response features are [7, 8, 9], they can be normalized to the range [0, 1] using min-max normalization and then fused using vector concatenation to obtain the load feature set [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9].

[0056] Step S300: inputting the load feature set into the pre-trained reinforcement learning strategy network, performing charge and discharge strategy priority matching processing on the load feature set through the reinforcement learning strategy network, and generating a priority ranking result of the target charge and discharge strategy.

[0057] The pre-trained reinforcement learning policy network is a pre-trained neural network model used to learn the optimal charge and discharge strategy for different load characteristics. The charge and discharge strategy priority matching process matches the load characteristic set with historical charge and discharge strategies and evaluates the priority of each charge and discharge strategy. The target charge and discharge strategy is the charge and discharge strategy that is matched to the current load characteristics. The priority ranking result is the result of sorting the target charge and discharge strategies from high to low priority.

[0058] As an embodiment, step S300 performs charge and discharge strategy priority matching processing on the load feature set through a reinforcement learning strategy network to generate a priority ranking result of the target charge and discharge strategy, which may specifically include the following steps S310 to S350:

[0059] Step S310: Obtaining a strategy execution record in a historical charge and discharge strategy set, wherein the strategy execution record includes a load matching parameter, a battery loss parameter, and a response time parameter of each historical charge and discharge strategy.

[0060] The historical charge and discharge strategy set refers to the set of all charge and discharge strategies used in the past. The strategy execution record is data that records the execution status of each historical charge and discharge strategy, including load matching parameters, battery loss parameters, and response time parameters. The load matching parameters refer to the degree of matching between the historical charge and discharge strategy and the load characteristics, reflecting the applicability of the strategy to the current load. The battery loss parameters refer to the loss of the battery during the execution of the historical charge and discharge strategy, such as capacity decay, internal resistance increase, etc. The response time parameter refers to the response speed of the historical charge and discharge strategy to load changes, reflecting the timeliness of the strategy. To obtain the strategy execution record, relevant data can be queried from the database. For example, in the battery management system of a supercomputing center, the execution record of each charge and discharge strategy is stored in the database. By querying the database, the strategy execution records in the historical charge and discharge strategy set can be obtained.

[0061] Step S320: constructing a strategy matching weight coefficient according to the load matching parameter, and performing reverse normalization processing on the battery loss parameter and the response time parameter to generate a strategy optimization constraint coefficient.

[0062] The strategy matching weight coefficient is constructed based on the load matching parameters and is used to measure the importance of each historical charge and discharge strategy in priority matching. Reverse normalization processes the battery loss parameters and response time parameters to align their value ranges with the load matching parameters, facilitating subsequent calculations. The strategy optimization constraint coefficient is constructed based on the reverse normalized battery loss parameters and response time parameters. It is used to constrain the selection of charge and discharge strategies, avoiding the selection of strategies that have excessive battery loss or excessive response time.

[0063] In specific implementation, each historical charge-discharge strategy is first assigned a weight based on the load matching parameter. A larger weight indicates a better match between the strategy and the current load. Next, the battery loss parameter and the response time parameter are denormalized, for example, using the maximum-minimum normalization method to normalize them to the range [0, 1]. Finally, the denormalized battery loss parameter and response time parameter are weighted and summed to obtain the strategy optimization constraint coefficient. For example, if the load matching parameters are [0.8, 0.6, 0.4], the battery loss parameters are [0.2, 0.3, 0.4], and the response time parameters are [0.1, 0.2, 0.3], after reverse normalization, the reverse normalized battery loss parameters are [0.8, 0.7, 0.6], and the response time parameters are [0.9, 0.8, 0.7]. If the weights are 0.5 and 0.5 respectively, the policy optimization constraint coefficients are [0.8×0.5+0.9×0.5, 0.7×0.5+0.8×0.5, 0.6×0.5+0.7×0.5]=[0.85, 0.75, 0.65].

[0064] Step S330: The strategy evaluation module in the reinforcement learning strategy network performs strategy matching prediction on the load feature set and outputs an initial priority score for each candidate charge and discharge strategy.

[0065] The policy evaluation module, a submodule within the reinforcement learning policy network, evaluates the match between each candidate charging and discharging policy and the load profile. Policy match prediction predicts the degree of match between each candidate charging and discharging policy and the load profile. The initial priority score is a preliminary evaluation score for each candidate charging and discharging policy, reflecting its priority given the current load profile.

[0066] As an embodiment, step S330 performs strategy matching prediction on the load feature set through the strategy evaluation module in the reinforcement learning strategy network, and outputs the initial priority score of each candidate charging and discharging strategy. Specifically, the following steps S331 to S337 may be included:

[0067] Step S331: performing a process of separating the steady-state and transient components of the current variation trend characteristics in the load characteristic set, and extracting the steady-state current parameter sequence and the transient current impact characteristics.

[0068] The separation of steady-state and transient components involves decomposing the current variation trend characteristics into steady-state and transient components. The steady-state current parameter sequence is the steady-state portion of the current variation trend characteristics, reflecting the long-term stable variation trend of the current. The transient current impact characteristic is the transient portion of the current variation trend characteristics, reflecting the sudden change in current. In specific implementations, filtering methods, such as low-pass filters and high-pass filters, can be used to separate the current variation trend characteristics. A low-pass filter can filter out high-frequency components to obtain a steady-state current parameter sequence; a high-pass filter can filter out low-frequency components to obtain a transient current impact characteristic. For example, using a Butterworth low-pass filter and a high-pass filter, with appropriate cutoff frequencies set, the current variation trend characteristics can be filtered to obtain a steady-state current parameter sequence and a transient current impact characteristic.

[0069] Step S332: performing weight distribution processing on the main energy distribution parameters and the secondary fluctuation component parameters in the voltage fluctuation pattern characteristics to generate a voltage fluctuation sensitivity coefficient.

[0070] The primary energy distribution parameter represents the portion of the voltage fluctuation pattern with the largest energy contribution, reflecting the primary characteristics of the voltage fluctuation. The secondary fluctuation component parameter represents the portion of the voltage fluctuation pattern with the smallest energy contribution, reflecting the secondary characteristics of the voltage fluctuation. The weight allocation process assigns different weights to the primary energy distribution parameter and the secondary fluctuation component parameter, then performs a weighted sum to obtain the voltage fluctuation sensitivity coefficient. In specific implementations, different weights can be assigned to the two parameters based on actual conditions. For example, if the primary energy distribution parameter is 0.8 and the secondary fluctuation component parameter is 0.2, with weights of 0.7 and 0.3, respectively, the voltage fluctuation sensitivity coefficient is 0.8 × 0.7 + 0.2 × 0.3 = 0.62.

[0071] Step S333: Based on the cumulative response energy distribution characteristics of the temperature-related response characteristics, thermal effect compensation processing is performed on the steady-state current parameter sequence to generate a temperature-calibrated steady-state current sequence.

[0072] Thermal effect compensation takes into account the impact of temperature on current and modifies the steady-state current parameter sequence. The cumulative response energy distribution characteristic reflects the cumulative response of temperature changes to load fluctuations. Based on this characteristic, the degree of temperature's impact on current can be determined. The temperature-calibrated steady-state current sequence is the steady-state current sequence obtained after thermal effect compensation, and more accurately reflects the current changes under temperature influence. In specific implementation, a thermal effect compensation model can be established. Based on the cumulative response energy distribution characteristic and the steady-state current parameter sequence, the temperature compensation for current is calculated. This compensation is then added to the steady-state current parameter sequence to obtain the temperature-calibrated steady-state current sequence. For example, if the thermal effect compensation model is I_calibrated = I_steady + k × E, where I_calibrated is the temperature-calibrated steady-state current sequence, I_steady is the steady-state current parameter sequence, k is the compensation coefficient, and E is the cumulative response energy distribution characteristic.

[0073] Step S334: performing time-series superposition processing on the transient current impact characteristics and the temperature-calibrated steady-state current sequence to generate a composite current characteristic vector.

[0074] Time series superposition processing superimposes the transient current surge signature and the temperature-calibrated steady-state current sequence on the time series to produce a comprehensive composite current feature vector. This composite current feature vector incorporates both steady-state and transient current information, providing a more comprehensive picture of current fluctuations. In practice, the composite current feature vector can be obtained by adding the data at corresponding time points of the transient current surge signature and the temperature-calibrated steady-state current sequence. For example, if the transient current surge signature is [1, 2, 3] and the temperature-calibrated steady-state current sequence is [4, 5, 6], the composite current feature vector is [1+4, 2+5, 3+6] = [5, 7, 9].

[0075] Step S335: performing cross-dimensional correlation analysis on the composite current characteristic vector and the voltage fluctuation sensitivity coefficient to construct a current-voltage joint fluctuation characteristic matrix.

[0076] Cross-dimensional correlation analysis analyzes the correlation between the composite current eigenvector and the voltage fluctuation sensitivity coefficient, taking into account their interactions across different dimensions. A current-voltage joint fluctuation characteristic matrix is ​​constructed based on the results of the cross-dimensional correlation analysis and is used to describe the joint fluctuation characteristics of current and voltage. In practice, correlation analysis methods, such as the Pearson correlation coefficient, can be used to calculate the correlation between the composite current eigenvector and the voltage fluctuation sensitivity coefficient. The correlation results are then combined into a matrix. For example, if the composite current eigenvector is [5, 7, 9] and the voltage fluctuation sensitivity coefficient is 0.62, the Pearson correlation coefficient between them is calculated to obtain a correlation coefficient matrix, which serves as the current-voltage joint fluctuation characteristic matrix.

[0077] Step S336: Input the current-voltage joint fluctuation feature matrix into the feature matching layer of the strategy evaluation module, perform a similarity comparison with the load adaptation feature template in the historical strategy execution record, and generate an initial matching score for the candidate charge and discharge strategy.

[0078] The feature matching layer is a layer in the strategy evaluation module, which is used for feature matching. The load adaptation feature template is a template related to the load characteristics in the historical strategy execution record, which reflects the characteristics of different charging and discharging strategies under different load conditions. Similarity comparison is to compare the current-voltage joint fluctuation feature matrix with the load adaptation feature template and calculate the similarity between them. The initial matching score is the score obtained based on the similarity comparison result, which reflects the degree of matching between the candidate charging and discharging strategy and the historical strategy in terms of load characteristics. In specific implementation, the similarity between the current-voltage joint fluctuation feature matrix and the load adaptation feature template can be calculated using methods such as Euclidean distance and cosine similarity. For example, the Euclidean distance is used to calculate the distance between the two matrices. The smaller the distance, the higher the similarity. The similarity is converted into a score to obtain the initial matching score.

[0079] Step S337: performing policy conflict detection processing on the initial matching score, eliminating policy matching results that are incompatible with the current battery status parameter set, and outputting an initial priority score.

[0080] The strategy conflict detection process is to check whether the candidate charge and discharge strategy corresponding to the initial matching score is compatible with the current battery state parameter set. If a candidate charge and discharge strategy will cause the battery to overcharge, over-discharge or other abnormal conditions, then the strategy is considered to be incompatible with the current battery state parameter set. The result of eliminating incompatible strategy matching is to set the initial matching score of the candidate charge and discharge strategy that is incompatible with the current battery state parameter set to a lower value or directly eliminate it. The initial priority score is the score output after the strategy conflict detection process, which more accurately reflects the priority of each candidate charge and discharge strategy under the current battery state. In specific implementation, some constraints can be set according to the battery state parameter set, such as the battery's state of charge range, voltage range, etc., to check whether each candidate charge and discharge strategy meets these constraints, and process the strategies that do not meet the constraints.

[0081] Step S340: performing a weighted operation on the initial priority score, the policy matching weight coefficient, and the policy optimization constraint coefficient to generate a revised policy priority score.

[0082] The weighted calculation is to sum the initial priority score, the policy matching weight coefficient, and the policy optimization constraint coefficient according to the set weights to obtain the corrected policy priority score. The corrected policy priority score comprehensively considers factors such as load matching, battery loss, and response time, and more comprehensively evaluates the priority of each candidate charging and discharging policy. In specific implementation, if the initial priority score, policy matching weight coefficient, and policy optimization constraint coefficient are S_init, W_match, and C_constraint, respectively, and the weights are w1, w2, and w3, respectively, then the corrected policy priority score S_corrected = w1 × S_init + w2 × W_match + w3 × C_constraint.

[0083] Step S350: Arrange the candidate charging and discharging strategies in descending order according to the revised strategy priority scores to generate a priority ranking result.

[0084] Descending sorting sorts candidate charging and discharging strategies from high to low based on their revised priority scores. The priority sorting result is the order of candidate charging and discharging strategies after descending sorting, reflecting the priority of each strategy based on the current load characteristics and battery status. In implementation, sorting algorithms such as quick sort and merge sort can be used to sort candidate charging and discharging strategies. For example, using the sorted function in Python, we can sort candidate charging and discharging strategies in descending order based on their revised priority scores to obtain the priority sorting result.

[0085] Step S400: adjusting the current charge and discharge strategy set according to the priority sorting result to generate an optimized charge and discharge strategy set; each charge and discharge strategy in the optimized charge and discharge strategy set includes a charge and discharge trigger condition parameter and a power allocation parameter.

[0086] The current charge and discharge strategy set refers to the set of charge and discharge strategies currently in use. Adjustment involves selecting and modifying strategies within the current charge and discharge strategy set based on the priority sorting results to improve strategy performance. The optimized charge and discharge strategy set is the set of charge and discharge strategies obtained after adjustment. Each charge and discharge strategy in the set includes charge and discharge trigger condition parameters and power allocation parameters. Charge and discharge trigger condition parameters refer to the conditions that trigger charge and discharge operations, such as the battery's state of charge and voltage. Power allocation parameters refer to the amount of power allocated to the battery during the charge and discharge process.

[0087] As an implementation manner, step S400 adjusts the current charge and discharge strategy set according to the priority sorting result to generate an optimized charge and discharge strategy set, which may specifically include the following steps S410 to S440:

[0088] Step S410: Filtering target strategy subsets with priority scores higher than a preset threshold from the priority sorting results, and extracting charge and discharge trigger condition parameters in the target strategy subsets.

[0089] The preset threshold is a pre-set priority score threshold used to filter out higher-priority policies. The target policy subset is the set of policies selected from the priority sorting results whose priority scores exceed the preset threshold. The charge and discharge trigger parameters are the charge and discharge trigger conditions for each policy in the target policy subset, such as the battery's state of charge threshold and voltage threshold. In implementation, the priority sorting results are traversed, and policies with priority scores above the preset threshold are added to the target policy subset. The charge and discharge trigger parameters are then extracted from the target policy subset. For example, if the preset threshold is 0.8 and the priority sorting results are [(policy 1, 0.9), (policy 2, 0.7), (policy 3, 0.85)], the target policy subset is [(policy 1, 0.9), (policy 3, 0.85)], and the extracted charge and discharge trigger parameters are the charge and discharge trigger conditions for policies 1 and 3.

[0090] Step S420: calibrating the charge and discharge trigger condition parameters according to the current change trend characteristics in the load characteristic set to generate a calibrated trigger condition parameter set.

[0091] Calibration adjusts the charge and discharge trigger parameters based on current trend characteristics to better adapt them to the current load conditions. The calibrated trigger parameter set is the set of charge and discharge trigger parameters obtained after calibration, which more accurately reflects the charge and discharge trigger conditions under the current load characteristics.

[0092] As an embodiment, step S420 calibrates the charge and discharge trigger condition parameters according to the current change trend characteristics in the load characteristic set to generate a calibrated trigger condition parameter set, which may specifically include the following steps S421 to S426:

[0093] Step S421: extracting the fluctuation period feature and the peak interval distribution feature from the current variation trend feature, and generating a current stability parameter based on the period stability coefficient of the fluctuation period feature and the discreteness of the peak interval distribution feature.

[0094] The period stability coefficient of the fluctuation period characteristic measures the stability of the fluctuation period and reflects the degree of variability of the fluctuation period. The dispersion of the peak interval distribution characteristic measures the degree of dispersion of the peak interval distribution and reflects the regularity of peak occurrence. The current stability parameter is constructed based on the period stability coefficient and dispersion and is used to describe the stability of the current. In specific implementation, the period stability coefficient and dispersion can be weighted and summed to obtain the current stability parameter. For example, if the period stability coefficient is 0.8 and the dispersion is 0.2, with weights of 0.7 and 0.3, respectively, the current stability parameter is 0.8 × 0.7 + 0.2 × 0.3 = 0.62.

[0095] Step S422: determining an adjustment ratio of the charge and discharge trigger threshold according to a difference between the current stability parameter and a preset stability reference value, and performing linear scaling processing on the current threshold in the charge and discharge trigger condition parameter based on the adjustment ratio to generate a calibrated current trigger threshold.

[0096] The preset stability reference value is a pre-set current stability reference value. The adjustment ratio is determined based on the difference between the current stability parameter and the preset stability reference value, and is used to adjust the charge and discharge trigger threshold. The linear scaling process is to scale the current threshold in the charge and discharge trigger condition parameters according to the adjustment ratio to obtain a calibrated current trigger threshold. In specific implementation, the difference between the current stability parameter and the preset stability reference value is first calculated, and then the adjustment ratio is determined based on the difference. For example, the larger the difference, the larger the adjustment ratio. Finally, the current threshold in the charge and discharge trigger condition parameters is multiplied by the adjustment ratio to obtain the calibrated current trigger threshold. For example, if the preset stability reference value is S_base, the current stability parameter is S_current, and the adjustment ratio is α, then ΔS=|S_current - S_base|, α=1 - kΔS (k is the attenuation coefficient).

[0097] Step S423: extracting the energy ratio relationship between the main energy distribution parameter and the secondary fluctuation component parameter in the voltage fluctuation pattern characteristics, and based on the energy ratio relationship, expanding the voltage fluctuation tolerance in the charge and discharge trigger condition parameters to generate an extended voltage tolerance parameter.

[0098] The energy ratio between the primary energy distribution parameter and the secondary fluctuation component parameter reflects the ratio between the primary and secondary characteristics of voltage fluctuation. The voltage fluctuation tolerance is the allowable voltage fluctuation range within the charge and discharge trigger condition parameters. Range expansion expands the voltage fluctuation tolerance based on the energy ratio to accommodate varying voltage fluctuations. The extended voltage tolerance parameter, resulting from range expansion, provides more flexible adaptation to changes in voltage fluctuations. In implementation, the expansion factor is determined based on the energy ratio between the primary energy distribution parameter and the secondary fluctuation component parameter. The voltage fluctuation tolerance is then multiplied by the expansion factor to obtain the extended voltage tolerance parameter. For example, if the energy ratio between the primary energy distribution parameter and the secondary fluctuation component parameter is 2:1, the voltage fluctuation tolerance is ±5V, and the expansion factor is 1.2, then the extended voltage tolerance parameter is ±5 × 1.2 = ±6V.

[0099] Step S424: performing parameter synchronization verification processing on the calibrated current trigger threshold and the extended voltage tolerance parameter, detecting whether the joint effective interval of the current threshold and the voltage tolerance covers the fluctuation range of the load fluctuation parameter set, and generating a conflict-free trigger parameter combination.

[0100] The parameter synchronization verification process is to check whether the calibration current trigger threshold and the extended voltage tolerance parameter can be effective simultaneously to avoid conflicts. The joint effective interval refers to the interval in which the calibration current trigger threshold and the extended voltage tolerance parameter are satisfied at the same time. The fluctuation range of the load fluctuation parameter set refers to the range of changes in current and voltage during the operation of the load. A conflict-free trigger parameter combination is a combination of the calibration current trigger threshold and the extended voltage tolerance parameter obtained after verification, which can effectively trigger the charge and discharge operation without conflict. In specific implementation, the calibration current trigger threshold and the extended voltage tolerance parameters are compared within the fluctuation range of the load fluctuation parameter set to check whether their joint effective interval covers the fluctuation range, and the parameters that do not meet the conditions are adjusted until a conflict-free trigger parameter combination is obtained.

[0101] Step S425: performing thermal inertia compensation processing on the non-conflicting trigger parameter combinations based on the cumulative response energy distribution characteristics of the temperature-related response characteristics, correcting the influence of temperature delay on the timing of trigger condition effectiveness, and generating a temperature-corrected trigger parameter set.

[0102] Thermal inertia compensation considers the impact of temperature on the timing of trigger conditions and corrects conflict-free trigger parameter combinations. The cumulative response energy distribution characteristic reflects the cumulative response of temperature changes to load fluctuations. Based on this characteristic, the degree of impact of temperature delay on the timing of trigger conditions can be determined. The temperature-corrected trigger parameter set is a set of trigger parameters obtained after thermal inertia compensation and more accurately reflects the trigger conditions under temperature considerations. In specific implementation, a thermal inertia compensation model can be established. Based on the cumulative response energy distribution characteristics and conflict-free trigger parameter combinations, the compensation amount for temperature delay on the timing of trigger conditions is calculated. The compensation amount is then added to the trigger conditions to obtain the temperature-corrected trigger parameter set. For example, if the thermal inertia compensation model is T_corrected = T_uncorrected + k × E, where T_corrected is the temperature-corrected trigger parameter set, T_uncorrected is the conflict-free trigger parameter combination, k is the compensation coefficient, and E is the cumulative response energy distribution characteristic.

[0103] Step S426: for the temperature correction trigger parameter set, filter out trigger condition parameters that match the fluctuation frequency and amplitude in the load fluctuation parameter set, and generate a calibrated trigger condition parameter set.

[0104] Screening involves selecting trigger condition parameters from the temperature-corrected trigger parameter set that match the fluctuation frequency and amplitude in the load fluctuation parameter set. Fluctuation frequency and amplitude are important characteristics of load fluctuation, and selecting trigger condition parameters that match them can improve the effectiveness of the charge and discharge strategy. The calibrated trigger condition parameter set is a filtered trigger condition parameter set that better matches the current load fluctuations. In specific implementation, the trigger condition parameters in the temperature-corrected trigger parameter set are compared with the fluctuation frequency and amplitude in the load fluctuation parameter set, and the trigger condition parameters with the highest degree of match are selected to form the calibrated trigger condition parameter set.

[0105] Step S430: performing load balancing verification on the power distribution parameters. If it is detected that the matching degree between the power distribution parameters and the load fluctuation parameter set is lower than a preset standard, performing nonlinear compensation processing on the power distribution parameters based on the voltage fluctuation pattern characteristics.

[0106] Load balancing verification is to check whether the power distribution parameters can evenly distribute the load in different time periods to avoid the situation where the load is too large or too small. The matching degree refers to the similarity between the power distribution parameters and the load fluctuation parameter set, which reflects the rationality of the power distribution. The preset standard is a pre-set matching threshold used to determine whether the power distribution is reasonable. Nonlinear compensation processing is to adjust the power distribution parameters according to the voltage fluctuation pattern characteristics to improve the matching degree between the power distribution and the load fluctuation. In specific implementation, the matching degree of the power distribution parameters and the load fluctuation parameter set is calculated. If the matching degree is lower than the preset standard, the power distribution parameters are nonlinearly adjusted according to the main energy distribution parameters and the secondary fluctuation component parameters in the voltage fluctuation pattern characteristics, such as using a nonlinear function for correction.

[0107] Step S440: performing combined optimization on the calibrated trigger condition parameter set and the compensated power allocation parameter to generate an optimized charge and discharge strategy set; the combined optimization includes parameter synchronization verification and conflict elimination processing.

[0108] Combination optimization involves combining the calibrated trigger condition parameter set and the compensated power allocation parameters to generate the optimal charge and discharge strategy set. Parameter synchronization verification checks whether the calibrated trigger condition parameter set and the compensated power allocation parameters can be synchronized to avoid conflicts. Conflict resolution adjusts conflicting parameters to ensure the effectiveness of the optimized charge and discharge strategy set. In specific implementation, the calibrated trigger condition parameter set and the compensated power allocation parameters are combined, followed by parameter synchronization verification and conflict resolution. Parameters that do not meet the requirements are adjusted until the optimized charge and discharge strategy set is achieved.

[0109] Step S500: performing charge and discharge operations based on the optimized charge and discharge strategy set, and collecting battery performance feedback data after the charge and discharge operations to iteratively update the strategy matching parameters of the reinforcement learning strategy network.

[0110] The optimized charge and discharge strategy set is a set of adjusted and optimized charge and discharge strategies. Executing a charge and discharge operation involves charging and discharging the battery according to the strategies in the optimized charge and discharge strategy set. Battery performance feedback data is collected after the charge and discharge operation, reflecting changes in battery performance, such as battery capacity, internal resistance, and temperature. Iterative updates are performed based on battery performance feedback data to improve the policy network's performance by updating the policy matching parameters of the reinforcement learning policy network.

[0111] As an embodiment, step S500, performing charge and discharge operations based on the optimized charge and discharge strategy set, and collecting battery performance feedback data after the charge and discharge operations to iteratively update the strategy matching parameters of the reinforcement learning strategy network, may specifically include the following steps S510 to S540:

[0112] Step S510: When executing each charge and discharge strategy in the optimized charge and discharge strategy set, the deviation between the battery output power curve and the strategy expected power curve is monitored in real time.

[0113] The battery output power curve is a graph showing the actual power output of the battery over time during the execution of the charge and discharge strategy. The strategy-expected power curve is a graph showing the expected power over time for each strategy in the optimized charge and discharge strategy set. Deviation refers to the degree of difference between the battery output power curve and the strategy-expected power curve, reflecting the degree to which the actual charge and discharge conditions conform to the expected conditions. Real-time monitoring involves continuously collecting battery output power data during the charge and discharge process and calculating its deviation from the strategy-expected power curve. In specific implementation, a power sensor can be used to collect battery output power data in real time. Then, a curve fitting method can be used to obtain the battery output power curve. The difference between the battery output power curve and the strategy-expected power curve is calculated to obtain the deviation.

[0114] Step S520: If the deviation exceeds the preset tolerance range, the trigger strategy executes an abnormal mark and records the abnormal timestamp and environmental parameter snapshot corresponding to the deviation.

[0115] The preset tolerance range is a pre-set deviation threshold used to determine whether the charging and discharging operations are normal. The strategy execution exception mark is to mark the current charging and discharging strategy execution when the deviation exceeds the preset tolerance range, indicating that the strategy execution is abnormal. The abnormal timestamp is the time point when the deviation exceeds the preset tolerance range. Recording the abnormal timestamp can facilitate subsequent analysis and processing. The environmental parameter snapshot is the environmental parameter data collected at the abnormal time point, such as temperature, humidity, air pressure, etc. Environmental factors may affect the performance of the battery. Recording the environmental parameter snapshot helps analyze the cause of the abnormality. In specific implementation, compare the deviation with the preset tolerance range. If the deviation exceeds the preset tolerance range, the strategy execution exception mark is triggered, and the abnormal timestamp and environmental parameter snapshot are recorded.

[0116] Step S530: Perform attenuation coefficient analysis on the battery performance feedback data, extract the capacity attenuation rate and internal resistance change rate after the strategy is executed, and input the capacity attenuation rate and internal resistance change rate into the strategy effect evaluation model to generate a strategy effectiveness score.

[0117] Attenuation coefficient analysis analyzes battery performance feedback data to determine the battery's capacity decay and internal resistance changes. The capacity decay rate refers to the percentage decrease in battery capacity after executing a charge and discharge strategy, reflecting the degree of battery capacity decay. The internal resistance change rate refers to the percentage change in battery internal resistance after executing a charge and discharge strategy, reflecting the change in battery internal resistance. The strategy effectiveness evaluation model is a pre-trained model used to evaluate the effectiveness of charge and discharge strategies. The strategy effectiveness score is a score obtained by the strategy effectiveness evaluation model based on the capacity decay rate and internal resistance change rate, reflecting the long-term impact of the charge and discharge strategy on battery performance.

[0118] As an embodiment, in step S530, the attenuation coefficient analysis is performed on the battery performance feedback data to extract the capacity attenuation rate and internal resistance change rate after the strategy is executed. Specifically, the following steps S531 to S535 may be included:

[0119] Step S531: After executing each charge and discharge strategy, collect battery charge and discharge cycle times and capacity test data sequence.

[0120] The number of battery charge and discharge cycles refers to the number of charge and discharge cycles a battery undergoes while executing a charge and discharge strategy. Capacity test data is obtained by testing the battery capacity after each charge and discharge cycle, reflecting how the battery capacity changes with the number of charge and discharge cycles. This data can be collected using battery testing equipment, such as a battery charge and discharge tester. After each charge and discharge cycle, the battery capacity is measured and the number of charge and discharge cycles and capacity test data are recorded.

[0121] Step S532: Perform linear regression analysis on the capacity test data sequence, and calculate the capacity decrease slope corresponding to the unit cycle number as the capacity decay rate.

[0122] Linear regression analysis is a statistical method used to analyze the linear relationship between two variables. In this step, a linear regression analysis is performed using the capacity test data sequence as the dependent variable and the number of charge and discharge cycles as the independent variable to obtain a linear equation for how the capacity changes with the number of charge and discharge cycles. The capacity decrease slope corresponding to the number of cycles is the slope of the linear equation, which reflects the proportion of the battery capacity decrease in each charge and discharge cycle, i.e., the capacity decay rate. In specific implementation, the `numpy.polyfit` function in Python can be used to perform linear regression analysis to obtain the coefficients of the linear equation, thereby calculating the capacity decay rate.

[0123] Step S533: measuring the DC internal resistance of the battery at different states of charge, and constructing an internal resistance-state of charge relationship curve.

[0124] The DC internal resistance value is the internal resistance of a battery in a DC circuit. Different states of charge (SOC) refer to the battery's state at different levels of charge or discharge. The internal resistance-State of Charge curve describes how the battery's internal resistance changes with SOC, reflecting the relationship between the battery's internal resistance and SOC. To measure the DC internal resistance of a battery at different SOCs, use an internal resistance tester. The internal resistance of the battery is measured at different SOCs, and the corresponding SOC and internal resistance values ​​are recorded. Then, using curve fitting methods, such as polynomial fitting, the internal resistance-State of Charge curve is constructed.

[0125] Step S534: calculating the curvature change of the internal resistance-state of charge relationship curve, and generating the internal resistance change rate according to the integration result of the curvature change.

[0126] The curvature change refers to the variation in curvature at different points on the internal resistance-State of Charge curve, reflecting the change in the curve's curvature. The integral result is obtained by integrating the curvature change within a set State of Charge range, reflecting the cumulative effect of changes in internal resistance with State of Charge. The internal resistance change rate is calculated based on this integral result and reflects the proportional change in the battery's internal resistance throughout the charge and discharge process. In implementation, the curvature change of the internal resistance-State of Charge curve is first calculated, using numerical differentiation methods. Next, the curvature change is integrated using a numerical integration method, such as the trapezoidal integration method or the Simpson integration method. Finally, the internal resistance change rate is calculated based on the integral result.

[0127] Step S535: normalize the capacity attenuation rate and the internal resistance change rate to generate a strategy effect evaluation input vector.

[0128] Normalization is the process of processing the capacity decay rate and the internal resistance change rate so that they have the same dimension and range, which is convenient for subsequent analysis and processing. The strategy effect evaluation input vector is a vector composed of the capacity decay rate and the internal resistance change rate obtained after normalization, which serves as the input of the strategy effect evaluation model. In specific implementation, the capacity decay rate and the internal resistance change rate can be normalized using the minimum-maximum normalization method or the Z-score normalization method. For example, using the minimum-maximum normalization method, the capacity decay rate and the internal resistance change rate are normalized to the range of [0, 1] to obtain the strategy effect evaluation input vector.

[0129] As an embodiment, in step S530, the capacity decay rate and the internal resistance change rate are input into the strategy effect evaluation model to generate a strategy effectiveness score, which may specifically include the following steps S536 to S5311:

[0130] Step S536: normalize the capacity decay rate by the amount of decay per unit time to generate a standardized capacity decay parameter; normalize the internal resistance change rate by the state of charge interval to generate a standardized internal resistance change parameter.

[0131] The normalization of the attenuation per unit time is to convert the capacity attenuation rate into the attenuation per unit time, and standardize it so that it has the same dimension and range. The standardized capacity attenuation parameter is the capacity attenuation parameter obtained after the normalization of the attenuation per unit time. The normalization of the state of charge interval is to normalize the internal resistance change rate within the state of charge interval so that it is comparable at different states of charge. The standardized internal resistance change parameter is the internal resistance change parameter obtained after the normalization of the state of charge interval. In specific implementation, for the capacity attenuation rate, it is divided by the charge and discharge time to obtain the attenuation per unit time, and then the minimum-maximum normalization or Z-score normalization method is used for normalization. For the internal resistance change rate, it is normalized within the state of charge interval so that its value range is between [0, 1].

[0132] Step S537: performing time series feature splicing processing on the standardized capacity decay parameter and the standardized internal resistance change parameter to generate a capacity-internal resistance joint evaluation vector.

[0133] Time series feature splicing processing is to splice the standardized capacity decay parameters and the standardized internal resistance change parameters in the time series to obtain a comprehensive vector. The capacity-internal resistance joint evaluation vector contains information on capacity decay and internal resistance change, and more comprehensively reflects the changes in battery performance. In specific implementation, the standardized capacity decay parameters and the standardized internal resistance change parameters are spliced ​​in time sequence to obtain the capacity-internal resistance joint evaluation vector. For example, if the standardized capacity decay parameters are [0.1, 0.2, 0.3] and the standardized internal resistance change parameters are [0.4, 0.5, 0.6], then the capacity-internal resistance joint evaluation vector is [0.1, 0.2, 0.3, 0.4, 0.5, 0.6].

[0134] Step S538: Input the capacity-internal resistance joint evaluation vector into the recurrent neural network layer in the strategy effect evaluation model to extract time-dependent features and output the attenuation pattern feature vector.

[0135] The recurrent neural network layer is a layer in the strategy effectiveness evaluation model that processes sequence data and can capture the temporal dependencies in the data. Temporal dependency feature extraction extracts time-related features from the capacity-internal resistance joint evaluation vector, reflecting the trend and pattern of battery performance changes. The decay pattern feature vector is a feature vector output after processing by the recurrent neural network layer, which contains the temporal pattern information of battery capacity decay and internal resistance changes. In specific implementation, the recurrent neural network layer can adopt structures such as long short-term memory networks (LSTMs) or gated recurrent units (GRUs). The capacity-internal resistance joint evaluation vector is input into the recurrent neural network layer, and after calculation and processing by the network, the decay pattern feature vector is output.

[0136] Step S539: Input the capacity-internal resistance joint evaluation vector into the convolution layer in the strategy effect evaluation model to capture the local fluctuation pattern and generate the internal resistance fluctuation feature vector.

[0137] The convolution layer is a layer in the strategy effect evaluation model that is used to extract local features from the data. Local fluctuation pattern capture is to extract local fluctuation patterns from the capacity-internal resistance joint evaluation vector, reflecting the changes in battery performance within a local range. The internal resistance fluctuation feature vector is a feature vector output after processing by the convolution layer, which contains the fluctuation information of the battery internal resistance within a local range. In specific implementation, the convolution layer can use a one-dimensional convolution kernel to perform a convolution operation on the capacity-internal resistance joint evaluation vector to extract local features, and then perform dimensionality reduction processing through the pooling layer to obtain the internal resistance fluctuation feature vector.

[0138] Step S5310: Perform cross-channel attention weight distribution processing on the attenuation mode feature vector and the internal resistance fluctuation feature vector to generate a strategy joint evaluation feature matrix.

[0139] The cross-channel attention weight allocation process performs a weighted combination of the decay pattern feature vector and the internal resistance fluctuation feature vector to highlight important feature information. The policy joint evaluation feature matrix is ​​the matrix obtained after the cross-channel attention weight allocation process. It combines the decay pattern characteristics and the internal resistance fluctuation characteristics to more comprehensively evaluate the effectiveness of the charging and discharging strategy. In specific implementation, the attention mechanism can be used to assign different weights to the decay pattern feature vector and the internal resistance fluctuation feature vector, and then perform a weighted sum to obtain the policy joint evaluation feature matrix. For example, using a multi-head attention mechanism, the attention weight of each feature vector is calculated, and then the feature vectors are weightedly combined according to the weight.

[0140] Step S5311: Input the strategy joint evaluation feature matrix into the fully connected layer in the strategy effect evaluation model for linear transformation processing to generate a strategy effectiveness score; the strategy effectiveness score is used to quantify the long-term impact of the charge and discharge strategy on battery performance.

[0141] The fully connected layer is the last layer in the policy effectiveness evaluation model. It is used to linearly transform the policy joint evaluation feature matrix, mapping it to a scalar value, the policy effectiveness score. The linear transformation process involves matrix multiplication of the policy joint evaluation feature matrix with the fully connected layer's weight matrix, and then adding a bias term to obtain the policy effectiveness score. The policy effectiveness score is a numerical value that quantifies the long-term impact of the charging and discharging strategy on battery performance. A higher score indicates a smaller impact on battery performance and a more effective strategy.

[0142] Step S540: Adjust the strategy matching parameters of the reinforcement learning strategy network according to the comparison result of the strategy effectiveness score and the preset score threshold; the adjustment includes updating the strategy priority weight and modifying the strategy matching rule.

[0143] The preset scoring threshold is a pre-set scoring criterion used to determine the effectiveness of a charge-discharge strategy. The comparison result is the relationship between the strategy effectiveness score and the preset scoring threshold. Adjustment involves modifying the policy matching parameters of the reinforcement learning policy network based on the comparison result to improve the policy network's performance. Policy priority weight update adjusts the priority weights of different charge-discharge strategies, giving higher priority to more effective strategies. Policy matching rule correction modifies the policy matching rules to improve the accuracy of policy matching. In specific implementation, if the policy effectiveness score is higher than the preset scoring threshold, the strategy is considered effective and its priority weight can be appropriately increased. If the policy effectiveness score is lower than the preset scoring threshold, the strategy is considered ineffective and the policy matching rules need to be revised to avoid selecting it. By continuously adjusting the policy matching parameters, the reinforcement learning policy network can learn more optimal charge-discharge strategies, improving battery performance and lifespan.

[0144] It is understandable that the various algorithms involved in the above-mentioned introduction of the embodiments of the present invention can be learned from the relevant content in the prior art. In order to save space, they will not be expanded too much in the embodiments of the present invention. In addition, when implementing the scheme of the present invention, those skilled in the art can supplement the details according to the common knowledge in this field. For example, according to the common knowledge in this field, normalization can be used to eliminate dimensional conflicts before feature fusion, interpolation can be used to eliminate dimensional differences, and historical data, experience or business scenario requirements can be combined to reasonably set thresholds, and models can be trained based on general model training methods, etc. The present invention will no longer provide redundant introductions to overly detailed implementation processes.

[0145] See also Figure 2 , Figure 2This is a schematic diagram of the structure of a computer system provided in an embodiment of the present invention. The computer system includes at least a processor 101, a communication interface 102, and a memory 103. The processor 101, communication interface 102, and memory 103 may be connected via a bus or other means. The processor 101 (also known as the Central Processing Unit (CPU)) is the computing and control core of the computer system, capable of parsing various instructions within the computer system and processing various data within the computer system. The communication interface 102 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), which can be used to send and receive data under the control of the processor 101. The communication interface 102 may also be used for data transmission and interaction within the computer system. The memory 103 is a storage device in the computer system for storing programs and data. It is understood that the memory 103 herein may include both the built-in memory of the computer system and, of course, the extended memory supported by the computer system. The memory 103 provides storage space, which stores the computer system's operating system, but this is not limited to this in the present invention.

[0146] In one embodiment, the processor 101 executes the supercomputing center battery charging and discharging strategy optimization method based on reinforcement learning provided in the above embodiment of the present invention by running the computer program in the memory 103.

Claims

1. A method for optimizing battery charging and discharging strategies in a supercomputing center based on reinforcement learning, characterized in that: include: Acquire real-time operating data sets of the supercomputing center battery system, wherein the real-time operating data sets include battery status parameter sets, load fluctuation parameter sets, and environmental monitoring parameter sets; Performing a time domain trend analysis on the current parameter sequence in the battery state parameter set, extracting the fluctuation period characteristics and peak interval distribution characteristics of the current parameter sequence, and performing a linear superposition process on the fluctuation period characteristics and the peak interval distribution characteristics to generate a current change trend characteristic; Performing frequency domain conversion processing on a voltage parameter sequence in the voltage fluctuation parameter set, extracting an energy ratio feature of a voltage fluctuation component below a first preset frequency and a noise component greater than a second preset frequency, and calculating a correlation between the energy ratio feature and a mean-variance feature of the voltage parameter sequence to generate a voltage fluctuation pattern feature; constructing a response delay function of temperature change to load fluctuation based on a temporal correlation between a temperature parameter sequence in the environmental monitoring parameter set and the load fluctuation parameter set, and generating a temperature-related response feature by integrating the response delay function; Performing multi-dimensional normalization fusion processing on the current change trend characteristics, voltage fluctuation pattern characteristics and temperature correlation response characteristics to output a load feature set; Inputting the load feature set into a pre-trained reinforcement learning strategy network, performing charge and discharge strategy priority matching processing on the load feature set through the reinforcement learning strategy network, and generating a priority ranking result of a target charge and discharge strategy; Adjusting the current charge and discharge strategy set according to the priority sorting result to generate an optimized charge and discharge strategy set; each charge and discharge strategy in the optimized charge and discharge strategy set includes a charge and discharge trigger condition parameter and a power allocation parameter; A charge and discharge operation is performed based on the optimized charge and discharge strategy set, and battery performance feedback data after the charge and discharge operation is collected to iteratively update the strategy matching parameters of the reinforcement learning strategy network.

2. The method according to claim 1, characterized in that The performing charge and discharge strategy priority matching processing on the load feature set by the reinforcement learning strategy network to generate a priority ranking result of a target charge and discharge strategy includes: Obtaining a strategy execution record in a historical charge and discharge strategy set, wherein the strategy execution record includes a load matching parameter, a battery loss parameter, and a response time parameter of each historical charge and discharge strategy; Constructing a strategy matching weight coefficient according to the load matching parameter, and performing reverse normalization processing on the battery loss parameter and the response time parameter to generate a strategy optimization constraint coefficient; Performing strategy matching prediction on the load feature set through a strategy evaluation module in the reinforcement learning strategy network, and outputting an initial priority score for each candidate charge and discharge strategy; Performing a weighted operation on the initial priority score, the strategy matching weight coefficient, and the strategy optimization constraint coefficient to generate a revised strategy priority score; The candidate charging and discharging strategies are arranged in descending order according to the revised strategy priority scores to generate the priority ranking result.

3. The method according to claim 1, characterized in that The step of adjusting the current charge and discharge strategy set according to the priority sorting result to generate an optimized charge and discharge strategy set includes: Filtering a target strategy subset having a priority score higher than a preset threshold from the priority sorting result, and extracting charge and discharge trigger condition parameters from the target strategy subset; calibrating the charge and discharge trigger condition parameters according to the current change trend characteristics in the load characteristic set to generate a calibrated trigger condition parameter set; Performing load balancing verification on the power allocation parameters, and if it is detected that the matching degree between the power allocation parameters and the load fluctuation parameter set is lower than a preset standard, performing nonlinear compensation processing on the power allocation parameters based on the voltage fluctuation pattern characteristics; The calibrated trigger condition parameter set and the compensated power allocation parameter are combined and optimized to generate the optimized charge and discharge strategy set; the combined optimization includes parameter synchronization verification and conflict elimination processing.

4. The method according to claim 1, wherein The performing of the charge and discharge operation based on the optimized charge and discharge strategy set, and collecting battery performance feedback data after the charge and discharge operation to iteratively update the strategy matching parameters of the reinforcement learning strategy network, includes: When executing each charge and discharge strategy in the set of optimized charge and discharge strategies, monitoring in real time the deviation between the battery output power curve and the strategy expected power curve; If the deviation exceeds the preset tolerance range, the trigger strategy executes an abnormal mark and records the abnormal timestamp and environmental parameter snapshot corresponding to the deviation; Performing attenuation coefficient analysis on the battery performance feedback data to extract the capacity attenuation rate and internal resistance change rate after the strategy is executed, and inputting the capacity attenuation rate and internal resistance change rate into a strategy effect evaluation model to generate a strategy effectiveness score; According to the comparison result of the strategy effectiveness score and the preset score threshold, the strategy matching parameters of the reinforcement learning strategy network are adjusted; the adjustment includes updating the strategy priority weight and modifying the strategy matching rule.

5. The method according to claim 1, characterized in that The performing time domain trend analysis on the current parameter sequence in the battery state parameter set to extract the fluctuation period characteristics and peak interval distribution characteristics of the current parameter sequence includes: Performing a sliding window segmentation process on the current parameter sequence to generate a plurality of current parameter subsequences; Perform local extreme value detection on each current parameter subsequence and extract the peak point position and the time interval data of adjacent peak points; Counting the distribution histogram of all time interval data, and calculating the kurtosis coefficient and skewness coefficient of the distribution histogram; generating the peak interval distribution feature according to the kurtosis coefficient and the skewness coefficient; Fourier transform processing is performed on the current parameter sequence to extract the main frequency component and its corresponding amplitude, and the fluctuation period feature is generated according to the inverse of the main frequency component.

6. The method according to claim 2, characterized in that The strategy evaluation module in the reinforcement learning strategy network performs strategy matching prediction on the load feature set and outputs an initial priority score for each candidate charge and discharge strategy, including: Separating the steady-state and transient components of the current change trend characteristics in the load feature set to extract the steady-state current parameter sequence and the transient current impact characteristics; Performing weight distribution processing on the main energy distribution parameters and the secondary fluctuation component parameters in the voltage fluctuation pattern characteristics to generate a voltage fluctuation sensitivity coefficient; Based on the cumulative response energy distribution characteristics of the temperature-related response characteristics, the steady-state current parameter sequence is subjected to thermal effect compensation processing to generate a temperature-calibrated steady-state current sequence; Performing time-series superposition processing on the transient current impact feature and the temperature-calibrated steady-state current sequence to generate a composite current feature vector; Performing cross-dimensional correlation analysis on the composite current characteristic vector and the voltage fluctuation sensitivity coefficient to construct a current-voltage joint fluctuation characteristic matrix; Inputting the current-voltage joint fluctuation feature matrix into the feature matching layer of the strategy evaluation module, performing a similarity comparison with the load adaptation feature template in the historical strategy execution record, and generating an initial matching score for the candidate charge and discharge strategy; A policy conflict detection process is performed on the initial matching score, policy matching results that are incompatible with the current battery state parameter set are eliminated, and the initial priority score is output.

7. The method according to claim 3, characterized in that The step of calibrating the charge and discharge trigger condition parameters according to the current change trend characteristics in the load characteristic set to generate a calibrated trigger condition parameter set includes: Extracting a fluctuation period feature and a peak interval distribution feature from the current change trend feature, and generating a current stability parameter based on a period stability coefficient of the fluctuation period feature and a discreteness of the peak interval distribution feature; determining an adjustment ratio of the charge and discharge trigger threshold according to a difference between the current stability parameter and a preset stability reference value, and performing linear scaling processing on the current threshold in the charge and discharge trigger condition parameter based on the adjustment ratio to generate a calibrated current trigger threshold; extracting an energy proportional relationship between a primary energy distribution parameter and a secondary fluctuation component parameter in the voltage fluctuation pattern characteristics, and based on the energy proportional relationship, expanding a range of a voltage fluctuation tolerance in the charge and discharge trigger condition parameter to generate an extended voltage tolerance parameter; Performing parameter synchronization verification processing on the calibrated current trigger threshold and the extended voltage tolerance parameter, detecting whether the joint effective interval of the current threshold and the voltage tolerance covers the fluctuation range of the load fluctuation parameter set, and generating a conflict-free trigger parameter combination; Performing thermal inertia compensation processing on the non-conflicting trigger parameter combination based on the cumulative response energy distribution characteristics of the temperature-related response characteristics, correcting the influence of temperature delay on the timing of trigger condition effectiveness to generate a temperature-corrected trigger parameter set; For the temperature correction trigger parameter set, trigger condition parameters that match the fluctuation frequency and amplitude in the load fluctuation parameter set are screened out to generate the calibrated trigger condition parameter set.

8. The method according to claim 4, characterized in that The performing of attenuation coefficient analysis on the battery performance feedback data to extract the capacity attenuation rate and internal resistance change rate after the strategy is executed includes: After executing each charge and discharge strategy, collect battery charge and discharge cycle times and capacity test data sequence; Performing a linear regression analysis on the capacity test data sequence to calculate the capacity decrease slope corresponding to the unit cycle number as the capacity decay rate; Measure the DC internal resistance of the battery at different states of charge and construct the internal resistance-state of charge relationship curve; Calculating a curvature change of the internal resistance-state of charge relationship curve, and generating the internal resistance change rate according to an integral result of the curvature change; The capacity attenuation rate and the internal resistance change rate are normalized to generate an input vector for strategy effect evaluation.

9. A computer system, characterized in that: include: a memory storing a computer program; A processor, configured to load the computer program to implement the method for optimizing the battery charging and discharging strategy of a supercomputing center based on reinforcement learning as described in any one of claims 1 to 8.

Images