Intelligent battery state regulation and control method based on deep Q network

Abstract

The invention discloses an intelligent battery state regulation and control method based on a deep Q network, and the method comprises the following steps: collecting the voltage, current, temperature and spatial distribution data of a battery, and generating a space-time working condition sequence; constructing a battery cell topological coupling graph based on the sequence, and obtaining an adjacent matrix and a Laplacian matrix to form an input set; extracting persistent coherence features, generating topological barcodes in different time windows and mapping the topological barcodes into embedded vectors to form a cross-scale set; establishing a Katyusha three-sequence parameter set, and inputting a cross-scale set; calculating a consistency loss constraint topological parameter; introducing distribution matching regularization to correct distribution difference in main parameter updating; generating a limited adversarial disturbance gradient correction momentum parameter in momentum parameter updating; and after iteration is completed, the depth Q network model is corrected, and battery charging and discharging and thermal management instructions are output. According to the method, topological characteristics, distribution matching and countermeasure disturbance are combined, and the regulation and control precision and stability of the battery system under complex working conditions are improved.

Description

Technical Field

[0001] This invention relates to the field of battery management and intelligent control technology, and in particular to a method for intelligent regulation of battery state based on deep Q-networks. Background Technology

[0002] With the development of the new energy industry, battery systems have been widely used in electric vehicles, energy storage power stations and distributed energy. The core task of battery management technology is to model and regulate the battery state by collecting parameters such as voltage, current, temperature and spatial distribution. However, existing technologies are mostly based on equivalent circuit models and statistical prediction methods. These methods can work in small-scale battery pack scenarios, but they are difficult to guarantee the accuracy of state estimation and the stability of control in large-scale battery packs and dynamic operating conditions.

[0003] Deep learning technology is gradually being applied to the field of battery management. Deep Q-networks have been introduced to handle decision-making problems in high-dimensional state spaces and complex environments. However, existing deep Q-network applications mainly focus on single-condition modeling and fail to introduce cell topology coupling relationships as input. This makes it impossible to fully characterize electrical connections and thermal coupling, resulting in insufficient effectiveness of control commands in multi-cell systems.

[0004] In terms of feature extraction, common methods rely on statistical features under fixed time windows and lack structured information across scales. Existing technologies do not utilize topological data analysis tools such as persistent cohomology to characterize the evolutionary features of short and long time windows. This deficiency makes it difficult for models to capture the long-term evolution of battery operating states, resulting in insufficient robustness of control strategies under complex operating conditions.

[0005] In the training process of deep Q-networks, existing methods rely on experience replay and target network update mechanisms, but do not consider the difference between historical data distribution and real-time data distribution. When the distribution difference is large, it can easily lead to slow convergence of the training process or even deviation from the optimal solution. At the same time, the momentum parameter update process has no robustness constraints and is easily affected by input disturbances, resulting in unstable control commands. The risk is more obvious in scenarios such as battery charging and discharging switching and dynamic changes in thermal management.

[0006] Therefore, how to provide a method for intelligent regulation of battery state based on deep Q-networks is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] One objective of this invention is to propose an intelligent battery state control method based on deep Q-networks. This invention fully utilizes algorithmic mechanisms such as topology data analysis, distribution matching regularization, and constrained adversarial perturbation. It describes in detail a method for correcting the deep Q-network model under battery spatiotemporal operating condition sequences by constructing a cell topology coupling graph, extracting cross-scale topology embeddings, and introducing a three-sequence parameter update framework based on an improved Katyusha algorithm. This method has the advantages of strong adaptability to complex battery system operating conditions, high accuracy of control commands, and good operational stability.

[0008] A method for intelligent regulation of battery state based on a deep Q-network according to an embodiment of the present invention includes the following steps:

[0009] The voltage, current, temperature and spatial distribution data of the battery are collected, and after synchronization and normalization processing, a spatiotemporal operating condition sequence is obtained.

[0010] A cell topology coupling graph is constructed based on spatiotemporal operating condition sequences, and a topological spatiotemporal input set is generated using adjacency matrix and Laplace matrix;

[0011] Calculate persistent homology features on the topological spatiotemporal input set, generate topological barcodes in short and long time windows and map them to topological embedding vectors to form a cross-scale topological embedding set;

[0012] Establish a three-sequence parameter set for the Katyusha algorithm, including principal parameters, momentum parameters, and topological parameters, and input a cross-scale topological embedding set;

[0013] In the iterative update, the cross-scale topology consistency loss is calculated, and the topology parameters are constrained based on the difference in topology barcodes in different time windows;

[0014] In the master parameter update, a distribution matching regularization is introduced to calculate the difference between the empirical replay distribution and the online distribution based on the maximum mean difference, and the result is used as a correction term.

[0015] In the momentum parameter update, a constrained adversarial perturbation gradient is generated based on the spatiotemporal condition sequence. The constrained adversarial perturbation gradient is then input into the momentum parameter update process to correct the momentum parameter.

[0016] After completing the three-sequence iterative update, the deep Q-network model is corrected using the updated principal parameters, momentum parameters, and topology parameters. Based on the corrected model, battery charging and discharging control commands and thermal management control commands are output.

[0017] Optionally, the process of constructing the cell topology coupling diagram based on the spatiotemporal operating condition sequence specifically includes:

[0018] Extract voltage data, current data, temperature data, and spatial distribution data from the spatiotemporal operating condition sequence;

[0019] Based on the spatial distribution data, a node number is assigned to each battery cell to obtain a set of battery cell nodes;

[0020] The electrical connection between battery cells is determined based on voltage and current data, and the thermal coupling between battery cells is determined based on temperature and spatial distribution data. Edges are established between battery cell nodes that have electrical or thermal connections.

[0021] Each edge is assigned a weight. The weight corresponding to electrical connection is calculated from voltage and current data, and the weight corresponding to thermal coupling is calculated from temperature and spatial distribution data.

[0022] In the node set, the degree information of each node is counted. The degree information represents the number of connections between the current node and other nodes.

[0023] By combining the node set, edge set, edge weight, and degree information, a topology description is generated.

[0024] The topological structure description is combined with the spatiotemporal operating condition sequence to form a topological spatiotemporal input set.

[0025] Optionally, the process of calculating persistent homology features on the topological spatiotemporal input set specifically includes:

[0026] The topological spatiotemporal input set is divided into short-time window data and long-time window data according to a fixed time span;

[0027] In the short time window data, feature vectors are extracted for each cell node. The feature vectors consist of voltage statistics, current statistics, temperature statistics, and spatial location information.

[0028] In short time window data, the similarity between nodes is calculated based on the node feature vectors, and the node connection relationship is established step by step according to the threshold to generate a topological filtering structure;

[0029] Persistent coherence features are extracted from the topological filtering structure of the short time window, the occurrence time index and disappearance time index of the topological features are recorded, and all persistent intervals are arranged to form a short time window topological barcode.

[0030] In long-term window data, persistent cohomological features are extracted using the same steps as in short-term window data to form long-term window topological barcodes;

[0031] Encode short-time-window topology barcodes into short-time-window topology embedding vectors, and encode long-time-window topology barcodes into long-time-window topology embedding vectors;

[0032] The short-time-window topological embedding vectors are combined with the long-time-window topological embedding vectors to form a cross-scale topological embedding set.

[0033] Optionally, the process of establishing the three-sequence parameter set for the Katyusha algorithm specifically includes:

[0034] Initialize the three-sequence parameter set, which includes the principal parameter, momentum parameter, and topology parameter;

[0035] In the main parameters, set the parameter vector used for value function updates; in the momentum parameters, set the parameter vector used for iteration acceleration; and in the topology parameters, set the parameter vector used for topology constraints.

[0036] Input the cross-scale topology embedding set into the topology parameters to establish an initial expression of the topology constraints;

[0037] The principal parameters, momentum parameters, and topological parameters are unified into the same iterative framework to form the initial state of the three-sequence parameter set.

[0038] Optionally, the process of calculating the cross-scale topological consistency loss in the iterative update specifically includes:

[0039] Determine the set of persistent intervals for short time windows based on short time window topology barcodes, and determine the set of persistent intervals for long time windows based on long time window topology barcodes;

[0040] Calculate the length and center position of each persistent interval for both the short-time window persistent interval set and the long-time window persistent interval set;

[0041] We construct a cross-scale consistency index by setting length difference weights and centrality difference weights, and define the topological consistency loss as follows:

[0042]

[0043] Where Δ represents the cross-scale topological consistency loss, K s K represents the number of short-term window duration intervals. l π represents the number of persistent intervals within a long-term window. ij This represents a matching indicator variable, taking a value of 0 or 1. A value of 1 indicates that the i-th persistence interval of the short-time window and the j-th persistence interval of the long-time window are matched, while a value of 0 indicates that no matching is established. This represents the length of the i-th persistent interval within the short time window. This represents the length of the j-th persistent interval within the long-term window. This indicates the center position of the i-th persistent interval within the short-time window. This represents the center position of the j-th persistent interval of the long-term window, where α is the length difference weight and β is the center difference weight.

[0044] The matching relationship is limited to a one-to-one correspondence. Each persistent interval of the short time window can match at most one persistent interval of the long time window, and each persistent interval of the long time window can match at most one persistent interval of the short time window. The matching indicator variable is limited to only 0 or 1.

[0045] The Δ input to the topology parameters is used as a cross-scale topology consistency loss in the iterative update to constrain the topology parameters.

[0046] Optionally, the process of introducing distribution matching regularization in the master parameter update specifically includes:

[0047] The historical spatiotemporal working condition sequence is stored as an experience replay set, and the working condition distribution is extracted from the experience replay set.

[0048] The spatiotemporal operating condition sequences collected in real time are used as an online set, and the operating condition distribution is extracted from the online set;

[0049] A distribution difference function is established between the distributions of the experience replay set and the online set. The distribution matching regularization term is defined using the maximum mean difference method:

[0050]

[0051] in, x represents the distribution matching regularization term. i Let y represent the i-th sample in the experience replay set. j Let represent the j-th sample in the online set, n represent the number of samples in the experience replay set, m represent the number of samples in the online set, and φ(·) represent the kernel mapping function;

[0052] After calculating the distribution matching regularization term, the distribution matching regularization term is used as a correction signal input into the master parameter update process;

[0053] During the master parameter update process, the distribution matching regularization term and the parameter update gradient are superimposed to adjust the master parameter vector.

[0054] Optionally, the process of generating constrained adversarial perturbation gradients in momentum parameter updates specifically includes:

[0055] Extract cell node features from the spatiotemporal operating condition sequence and combine them with the cell topology coupling graph to generate disturbance-resistant input;

[0056] The constrained adversarial gradient is defined on the adversarial perturbation input as follows:

[0057]

[0058] Among them, g advε represents the constrained adversarial perturbation gradient, ε represents the perturbation amplitude constraint constant, and x represents the input consisting of the spatiotemporal operating condition sequence and the cell topology coupling diagram. Let ρ represent the gradient operator with respect to input x. δ (·) represents the Huber loss with a threshold δ as a parameter, r represents the immediate reward, γ represents the discount factor, s represents the current state, a represents the current action, and s ′ Indicates the next state, a ′ Q represents the action variable used to calculate the target value in the next state. θ (s,a) represents the prediction function of the deep Q-network, expressed in terms of the parameter vector θ. θ -(s ′ ,a ′ ) represents the target network parameter vector θ - The prediction function, represented by the deep Q-network prediction function, is a neural network model controlled by a parameter vector. It receives features extracted from the spatiotemporal operating condition sequence and the cell topology coupling diagram at the input end and generates state-action value at the output end. The parameter vector is iteratively updated during the training process by minimizing the temporal difference loss, so that the prediction function gradually approaches the optimal state-action value function and provides a basis for the generation of battery charging and discharging control commands and thermal management control commands.

[0059] The constrained adversarial perturbation gradient is superimposed with the original gradient direction of the momentum parameter to form the momentum update direction;

[0060] The momentum parameter vector is adjusted according to the momentum update direction, and the momentum parameter correction is completed during the iteration process.

[0061] Optionally, the process of completing the three-sequence iterative update and using the update parameters to correct the deep Q-network model specifically includes:

[0062] After completing the iterative updates of the master parameters, momentum parameters, and topology parameters, a corrected deep Q-network model is formed. The deep Q-network model consists of an input layer, a hidden layer, and an output layer. The input layer receives features extracted from the spatiotemporal operating condition sequence and the cell topology coupling diagram. The hidden layer generates state feature representations through nonlinear mapping. The output layer calculates the correspondence value between the state and the action.

[0063] By inputting the spatiotemporal condition sequence into the modified deep Q-network model, the state-action value corresponding to each candidate action is obtained.

[0064] Select the action with the highest state-action value from all candidate actions and use it as the battery control output action.

[0065] The battery regulation output action is decomposed into battery charging and discharging control commands and battery thermal management control commands, and the control commands are sent to the battery control system for execution.

[0066] The beneficial effects of this invention are:

[0067] This invention constructs a cell topology coupling diagram based on a spatiotemporal operating condition sequence and extracts cross-scale topology embeddings. It introduces persistent coherence features to establish topology consistency constraints, thereby stably characterizing the electrical connection and thermal coupling relationships between cells at different time scales. This enables accurate characterization of the operating state of complex battery packs and ensures that the control model responds sensitively to state changes during charging, discharging, and thermal management processes.

[0068] In the training process of deep Q-network, a distribution matching regularization method is introduced to constrain the distribution difference between the experience playback set and the online set, avoiding the model deviation problem caused by the inconsistency between the distribution of historical data and real-time data. Furthermore, the robustness of the momentum parameter update process is enhanced by constrained adversarial perturbation gradient, effectively suppressing the instability risk caused by input perturbation. This enables the trained model to output reliable battery charging and discharging control commands and thermal management control commands under various operating conditions.

[0069] By incorporating cross-scale topological consistency loss, distribution matching regularization, and constrained adversarial perturbation gradient into the improved Katyusha three-sequence parameter iterative framework, a modified deep Q-network model is formed under the joint update of principal parameters, momentum parameters, and topological parameters. This model has stronger adaptability and stability in complex working conditions and enables intelligent control of the battery system. Attached Figure Description

[0070] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0071] Figure 1 This is a flowchart of a battery state intelligent control method based on a deep Q-network proposed in this invention;

[0072] Figure 2 This is a schematic diagram of the cell topology coupling structure of the intelligent battery state control method based on deep Q-network proposed in this invention.

[0073] Figure 3 This is a schematic diagram of the deep Q-network model structure of the intelligent battery state control method based on deep Q-network proposed in this invention. Detailed Implementation

[0074] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0075] refer to Figure 1-3 A method for intelligent regulation of battery state based on deep Q-networks includes the following steps:

[0076] The voltage, current, temperature and spatial distribution data of the battery are collected, and after synchronization and normalization processing, a spatiotemporal operating condition sequence is obtained.

[0077] A cell topology coupling graph is constructed based on spatiotemporal operating condition sequences, and a topological spatiotemporal input set is generated using adjacency matrix and Laplace matrix;

[0078] Calculate persistent homology features on the topological spatiotemporal input set, generate topological barcodes in short and long time windows and map them to topological embedding vectors to form a cross-scale topological embedding set;

[0079] Establish a three-sequence parameter set for the Katyusha algorithm, including principal parameters, momentum parameters, and topological parameters, and input a cross-scale topological embedding set;

[0080] In the iterative update, the cross-scale topology consistency loss is calculated, and the topology parameters are constrained based on the difference in topology barcodes in different time windows;

[0081] In the master parameter update, a distribution matching regularization is introduced to calculate the difference between the empirical replay distribution and the online distribution based on the maximum mean difference, and the result is used as a correction term.

[0082] In the momentum parameter update, a constrained adversarial perturbation gradient is generated based on the spatiotemporal condition sequence. The constrained adversarial perturbation gradient is then input into the momentum parameter update process to correct the momentum parameter.

[0083] After completing the three-sequence iterative update, the deep Q-network model is corrected using the updated principal parameters, momentum parameters, and topology parameters. Based on the corrected model, battery charging and discharging control commands and thermal management control commands are output.

[0084] This invention generates a spatiotemporal operating condition sequence by collecting battery voltage, current, temperature, and spatial distribution data and performing synchronization and normalization processing. This allows battery operating characteristics to be expressed in a unified temporal and spatial coordinate form, thereby avoiding inconsistencies caused by different data sources in subsequent modeling processes. It ensures the integrity and standardization of input features and provides high-quality basic data support for subsequent construction of topology structures and deep models, effectively improving the accuracy of battery state monitoring and control.

[0085] In this embodiment, the process of constructing the cell topology coupling diagram based on the spatiotemporal operating condition sequence specifically includes:

[0086] Extract voltage data, current data, temperature data, and spatial distribution data from the spatiotemporal operating condition sequence;

[0087] Based on the spatial distribution data, a node number is assigned to each battery cell to obtain a set of battery cell nodes;

[0088] The electrical connection between battery cells is determined based on voltage and current data, and the thermal coupling between battery cells is determined based on temperature and spatial distribution data. Edges are established between battery cell nodes that have electrical or thermal connections.

[0089] Each edge is assigned a weight. The weight corresponding to electrical connection is calculated from voltage and current data, and the weight corresponding to thermal coupling is calculated from temperature and spatial distribution data.

[0090] In the node set, the degree information of each node is counted. The degree information represents the number of connections between the current node and other nodes.

[0091] By combining the node set, edge set, edge weight, and degree information, a topology description is generated.

[0092] The topological structure description is combined with the spatiotemporal operating condition sequence to form a topological spatiotemporal input set.

[0093] This invention constructs a cell topology coupling graph based on a spatiotemporal operating condition sequence and generates a spatiotemporal topology input set using adjacency and Laplace matrices. This enables unified modeling of electrical connections and thermal coupling relationships between cells, allowing the system to simultaneously capture local coupling effects and overall topological features. This provides a more accurate and comprehensive input expression for battery state analysis and control, avoiding the shortcomings of traditional methods that model based only on single points or average values.

[0094] In this embodiment, the process of calculating persistent homology features on the topological spatiotemporal input set specifically includes:

[0095] The topological spatiotemporal input set is divided into short-time window data and long-time window data according to a fixed time span;

[0096] In the short time window data, feature vectors are extracted for each cell node. The feature vectors consist of voltage statistics, current statistics, temperature statistics, and spatial location information.

[0097] In short time window data, the similarity between nodes is calculated based on the node feature vectors, and the node connection relationship is established step by step according to the threshold to generate a topological filtering structure;

[0098] Persistent coherence features are extracted from the topological filtering structure of the short time window, the occurrence time index and disappearance time index of the topological features are recorded, and all persistent intervals are arranged to form a short time window topological barcode.

[0099] In long-term window data, persistent cohomological features are extracted using the same steps as in short-term window data to form long-term window topological barcodes;

[0100] Encode short-time-window topology barcodes into short-time-window topology embedding vectors, and encode long-time-window topology barcodes into long-time-window topology embedding vectors;

[0101] The short-time-window topological embedding vectors are combined with the long-time-window topological embedding vectors to form a cross-scale topological embedding set.

[0102] This invention calculates persistent cohomological features on the topological spatiotemporal input set under different time windows and generates topological barcodes that are mapped to topological embedding vectors to form a cross-scale embedding set. This achieves a unified expression of battery operating features on both short-term and long-term scales, enabling the model to simultaneously focus on instantaneous state fluctuations and long-term evolution trends. This improves the timeliness and stability of anomaly detection and control in battery systems, breaking through the limitation of traditional methods that rely on only a single time scale.

[0103] In this embodiment, the process of establishing the three-sequence parameter set of the Katyusha algorithm specifically includes:

[0104] Initialize the three-sequence parameter set, which includes the principal parameter, momentum parameter, and topology parameter;

[0105] In the main parameters, set the parameter vector used for value function updates; in the momentum parameters, set the parameter vector used for iteration acceleration; and in the topology parameters, set the parameter vector used for topology constraints.

[0106] Input the cross-scale topology embedding set into the topology parameters to establish an initial expression of the topology constraints;

[0107] The principal parameters, momentum parameters, and topological parameters are unified into the same iterative framework to form the initial state of the three-sequence parameter set.

[0108] This invention establishes a three-sequence parameter set for the Katyusha algorithm and uniformly inputs the principal parameter, momentum parameter, and topological parameter into a cross-scale topological embedding set, forming an iterative mechanism under the same optimization framework. This enables the simultaneous updating of the value function, accelerated convergence, and structural constraints, allowing the model to balance efficiency and stability during training, thereby improving the reliability and adaptability of parameter updates during intelligent battery state control.

[0109] In this embodiment, the process of calculating the cross-scale topological consistency loss during iterative updates specifically includes:

[0110] Determine the set of persistent intervals for short time windows based on short time window topology barcodes, and determine the set of persistent intervals for long time windows based on long time window topology barcodes;

[0111] Calculate the length and center position of each persistent interval for both the short-time window persistent interval set and the long-time window persistent interval set;

[0112] We construct a cross-scale consistency index by setting length difference weights and centrality difference weights, and define the topological consistency loss as follows:

[0113]

[0114] Where Δ represents the cross-scale topological consistency loss, K s K represents the number of short-term window duration intervals. l π represents the number of persistent intervals within a long-term window. ij This represents a matching indicator variable, taking a value of 0 or 1. A value of 1 indicates that the i-th persistence interval of the short-time window and the j-th persistence interval of the long-time window are matched, while a value of 0 indicates that no matching is established. This represents the length of the i-th persistent interval within the short time window. This represents the length of the j-th persistent interval within the long-term window. This indicates the center position of the i-th persistent interval within the short-time window. This represents the center position of the j-th persistent interval of the long-term window, where α is the length difference weight and β is the center difference weight.

[0115] The matching relationship is limited to a one-to-one correspondence. Each persistent interval of the short time window can match at most one persistent interval of the long time window, and each persistent interval of the long time window can match at most one persistent interval of the short time window. The matching indicator variable is limited to only 0 or 1.

[0116] The above formula is based on a binary matching model consisting of a set of persistent intervals with a short time window and a set of persistent intervals with a long time window. The reason is that persistent coherence features are represented in the form of persistent intervals, the matching cost can be quantified by the length difference and the center position difference, the matching relationship can be represented by an indicator variable that takes the value 0 or 1, and the total cost is obtained by summing all candidate matches, thus forming an optimized expression with the goal of minimizing the total cost.

[0117] The Δ input to the topology parameters is used as a cross-scale topology consistency loss in the iterative update to constrain the topology parameters.

[0118] This invention achieves consistency maintenance of topological features at both short-term and long-term scales by calculating cross-scale topological consistency loss during iterative updates and constraining topological parameters based on the difference in topological barcodes at different time windows. This enables the model to avoid feature shift problems caused by differences in time scales, thereby ensuring the coherence and interpretability of battery state regulation and effectively improving the model's adaptability to complex operating conditions.

[0119] In this embodiment, the process of introducing distribution matching regularization in the master parameter update specifically includes:

[0120] The historical spatiotemporal working condition sequence is stored as an experience replay set, and the working condition distribution is extracted from the experience replay set.

[0121] The spatiotemporal operating condition sequences collected in real time are used as an online set, and the operating condition distribution is extracted from the online set;

[0122] A distribution difference function is established between the distributions of the experience replay set and the online set. The distribution matching regularization term is defined using the maximum mean difference method:

[0123]

[0124] in, x represents the distribution matching regularization term. i Let y represent the i-th sample in the experience replay set. j Let represent the j-th sample in the online set, n represent the number of samples in the experience replay set, m represent the number of samples in the online set, and φ(·) represent the kernel mapping function. The above formula is constructed based on the difference in mean embedding between the experience replay set and the online set in the reproducing kernel Hilbert space. The reason is that the difference in distribution between the two sets can be represented by the distance metric of mean embedding, thus forming a computable distribution matching regularization term.

[0125] After calculating the distribution matching regularization term, the distribution matching regularization term is used as a correction signal input into the master parameter update process;

[0126] During the master parameter update process, the distribution matching regularization term and the parameter update gradient are superimposed to adjust the master parameter vector.

[0127] This invention introduces distribution matching regularization into the master parameter update and uses the difference between the empirical replay distribution and the online distribution calculated based on the maximum mean difference as a correction term input into the iteration process. This achieves distribution alignment between historical and real-time data, enabling parameter updates to maintain stability and reducing control errors caused by inconsistent data distributions. This improves the adaptability and robustness of the battery system under varying operating conditions.

[0128] In this embodiment, the process of generating a constrained adversarial perturbation gradient in momentum parameter updates specifically includes:

[0129] Extract cell node features from the spatiotemporal operating condition sequence and combine them with the cell topology coupling graph to generate disturbance-resistant input;

[0130] The constrained adversarial gradient is defined on the adversarial perturbation input as follows:

[0131]

[0132] Among them, g adv ε represents the constrained adversarial perturbation gradient, ε represents the perturbation amplitude constraint constant, and x represents the input consisting of the spatiotemporal operating condition sequence and the cell topology coupling diagram. Let ρ represent the gradient operator with respect to input x. δ (·) represents the Huber loss with a threshold δ as a parameter, r represents the immediate reward, γ represents the discount factor, s represents the current state, a represents the current action, and s ′ Indicates the next state, a ′ Q represents the action variable used to calculate the target value in the next state. θ (s,a) represents the prediction function of the deep Q-network, expressed in terms of the parameter vector θ. θ -(s ′ ,a ′ ) represents the target network parameter vector θ - The prediction function, represented by the deep Q-network prediction function, is a neural network model controlled by a parameter vector. At the input end, it receives features extracted from the spatiotemporal operating condition sequence and the cell topology coupling diagram. At the output end, it generates a state-action value. The parameter vector is iteratively updated during training by minimizing the temporal difference loss, so that the prediction function gradually approaches the optimal state-action value function and provides a basis for the generation of battery charging and discharging control commands and thermal management control commands. The above formula is based on solving the temporal difference loss in the ascending direction under the infinite norm constraint. The reason is that the goal is to maximize the growth of the temporal difference loss within a given amplitude limit, thereby obtaining the constrained adversarial perturbation gradient generated according to the gradient sign direction of the loss on the input.

[0133] The constrained adversarial perturbation gradient is superimposed with the original gradient direction of the momentum parameter to form the momentum update direction;

[0134] The momentum parameter vector is adjusted according to the momentum update direction, and the momentum parameter correction is completed during the iteration process.

[0135] This invention achieves momentum correction by generating a constrained adversarial gradient based on the spatiotemporal operating condition sequence during momentum parameter updates and inputting it into the update process. This enables the model to maintain a stable iterative direction when facing input disturbances or abnormal operating conditions, thereby enhancing the anti-interference capability of deep Q-networks in complex environments and avoiding convergence offset problems caused by external noise or data anomalies. This effectively improves the safety and reliability of intelligent battery state control.

[0136] In this embodiment, the process of completing the three-sequence iterative update and using the update parameters to correct the deep Q-network model specifically includes:

[0137] After completing the iterative updates of the master parameters, momentum parameters, and topology parameters, a corrected deep Q-network model is formed. The deep Q-network model consists of an input layer, a hidden layer, and an output layer. The input layer receives features extracted from the spatiotemporal operating condition sequence and the cell topology coupling diagram. The hidden layer generates state feature representations through nonlinear mapping. The output layer calculates the correspondence value between the state and the action.

[0138] By inputting the spatiotemporal condition sequence into the modified deep Q-network model, the state-action value corresponding to each candidate action is obtained.

[0139] Select the action with the highest state-action value from all candidate actions and use it as the battery control output action.

[0140] The battery regulation output action is decomposed into battery charging and discharging control commands and battery thermal management control commands, and the control commands are sent to the battery control system for execution.

[0141] This invention modifies the deep Q-network model by using the corrected principal parameters, momentum parameters, and topology parameters after completing a three-sequence iterative update, and outputs battery charging, discharging, and thermal management control commands. This achieves a closed-loop process from the original spatiotemporal conditions to the actual control commands, enabling the model to continuously output stable and effective control decisions under complex dynamic conditions, thereby improving the overall performance of the battery system in terms of energy efficiency, safe operation, and lifespan management.

[0142] Example 1:

[0143] To verify the feasibility of this invention in practice, it was applied to a battery energy storage system consisting of hundreds of lithium-ion cells. This system operates under high-power charging and discharging conditions, and the cells not only have obvious electrical connections but also complex thermal coupling relationships. Existing battery management systems mostly adopt traditional methods, namely, battery regulation based on equivalent circuit models and rule-based control strategies. In this traditional method, voltage, current, and temperature sensors are first used to collect real-time data of individual cells. Then, the battery's state of charge and health are estimated through a simple equivalent circuit model, and charging and discharging are managed based on fixed thresholds. For example, charging is immediately stopped when the cell voltage exceeds the upper limit threshold, and the fan is turned on to cool down when the temperature exceeds the threshold. In addition, the thermal management part relies on the average temperature value to control the switching of the cooling unit, lacking the ability to model the spatial distribution and coupling effects of cells. Although this method is simple to implement, it is difficult to accurately reflect the overall dynamic characteristics when the number of cells is large, often leading to local cell overheating, voltage unevenness, and energy waste.

[0144] When applying the method of this invention in this scenario, a spatiotemporal operating condition sequence is first constructed by monitoring the voltage, current, temperature, and spatial location of individual battery cells. A cell topology coupling graph is then established to characterize the electrical and thermal coupling relationships between the cells. The system maps the operating condition features and topological features into a topological spatiotemporal input set and extracts persistent coherence features at different time scales. A cross-scale topological embedding set is generated through persistent barcodes. Subsequently, in the improved Katyusha three-sequence iterative framework, the principal parameters, momentum parameters, and topological parameters are set respectively. Cross-scale topological consistency constraints ensure that the model accurately describes the cell correlation relationships at different time windows. To address the error caused by the inconsistency between historical and real-time data distributions, this invention introduces a distribution matching regularization term, enabling the model to maintain distribution alignment between empirical replay data and online data. In the momentum parameter update, a constrained adversarial perturbation gradient is also introduced to enhance the model's stability and anti-perturbation capability under abnormal operating conditions. Finally, the modified deep Q-network model can output battery charging and discharging control commands and thermal management control commands, achieving refined and intelligent battery regulation.

[0145] During the experiment, the traditional method was compared with the method of this invention. The traditional method, which relies solely on equivalent circuit models and threshold control, has limited performance in terms of charge and discharge efficiency, temperature equalization, and voltage consistency. The comparative data shows that in continuous charge and discharge cycles, the average energy utilization rate of the battery pack under the traditional method is about 85%, while it is improved to over 92% after applying the present invention. In terms of thermal management, the traditional method controls the cooling device through average temperature, resulting in the cell temperature difference being maintained at about 15 degrees Celsius. The present invention, however, utilizes topological coupling modeling and intelligent control to reduce the temperature difference to within 6 degrees Celsius. In terms of voltage consistency, the voltage deviation of a single cell under the traditional method is about 0.12 volts, while it is reduced to 0.05 volts after applying the present invention. In terms of control response speed, the traditional method, which relies on threshold detection and hysteresis control, has a response time of 1.8 seconds, while the present invention can complete the control action within 1.1 seconds.

[0146] To illustrate the differences between the two methods, the experimental data comparison is shown in Table 1:

[0147] Table 1 Comparison of Experimental Results of Battery State Control Methods

[0148]

[0149] The comparison results show that traditional methods are significantly inadequate under complex operating conditions due to their simple modeling and insufficient feature utilization. In contrast, this invention introduces topological homology features, distribution matching regularization, and constrained adversarial perturbations to optimize and update parameters within a deep Q-network framework. This results in significant improvements in energy efficiency, temperature balance, voltage consistency, and control response time of the battery system. It also effectively avoids system downtime caused by overheating of individual cells or excessive voltage deviation, verifying the feasibility and advancement of this invention in practical applications.

[0150] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1.A deep Q network based intelligent regulation method for battery state, characterized in that, The method comprises the following steps: Collecting voltage, current, temperature and spatial distribution data of the battery, synchronizing and normalizing the data to obtain a time-space working condition sequence; Based on the time-space working condition sequence, an electric cell topology coupling graph is constructed, and an adjacency matrix and a Laplacian matrix are used to generate a topology time-space input set; Persistent homology features are calculated for the topology time-space input set, and topology barcodes are generated in short and long time windows and mapped to topology embedding vectors to form a cross-scale topology embedding set; A three-sequence parameter set of the Katyusha algorithm is established, including main parameters, momentum parameters and topology parameters, and the cross-scale topology embedding set is inputted; In the iterative update, the cross-scale topology consistency loss is calculated, and the topology parameters are constrained according to the difference of the topology barcodes in different time windows; In the main parameter update, a distribution matching regularity is introduced, the difference between the experience replay distribution and the online distribution is calculated according to the maximum mean difference, and the result is used as a correction term; In the momentum parameter update, a limited adversarial perturbation gradient is generated based on the time-space working condition sequence, and the limited adversarial perturbation gradient is inputted into the momentum parameter update process to correct the momentum parameters; After completing the three-sequence iterative update, the main parameters, momentum parameters and topology parameters are updated, and the deep Q network model is corrected based on the updated model, and the battery charging and discharging control instructions and the thermal management control instructions are outputted. 2.The battery state intelligent regulation method based on deep Q network according to claim 1, wherein, The process of constructing the electric cell topology coupling graph based on the time-space working condition sequence specifically comprises: Extracting voltage data, current data, temperature data and spatial distribution data from the time-space working condition sequence; Establishing node numbers for each electric cell based on the spatial distribution data to obtain an electric cell node set; Judging the electrical connection relationship between the electric cells according to the voltage data and the current data, judging the thermal coupling relationship between the electric cells according to the temperature data and the spatial distribution data, and establishing edges between the electric cell nodes with the electrical connection relationship or the thermal coupling relationship; Assigning weights to each edge, the weight corresponding to the electrical connection relationship is calculated from the voltage data and the current data, and the weight corresponding to the thermal coupling relationship is calculated from the temperature data and the spatial distribution data; Counting the degree information of each node in the node set, which represents the connection number of the current node with other nodes; Generating a topology structure description by integrating the node set, the edge set, the edge weight and the degree information; Combining the topology structure description with the time-space working condition sequence to form a topology time-space input set. 3.The battery state intelligent regulation method based on deep Q network according to claim 1, wherein, The process of calculating the persistent homology features for the topology time-space input set specifically comprises: Dividing the short time window data and the long time window data from the topology time-space input set according to a fixed time span; Extracting a feature vector for each electric cell node in the short time window data, the feature vector being composed of voltage statistics, current statistics, temperature statistics and spatial position information; Calculating the similarity between the nodes in the short time window data according to the node feature vectors, gradually establishing the node connection relationship according to a threshold, and generating a topology filtered structure; Extracting persistent homology features from the topology filtered structure of the short time window, recording the appearance time index and the disappearance time index of the topology features, and arranging all the persistent intervals to form a short time window topology barcode; The persistent homology features are extracted in the long time window data in the same steps as in the short time window, forming long time window topological barcodes; The short time window topological barcodes are encoded into short time window topological embedding vectors, and the long time window topological barcodes are encoded into long time window topological embedding vectors; The short time window topological embedding vectors and the long time window topological embedding vectors are combined to form a cross-scale topological embedding set. 4.The battery state intelligent regulation method based on deep Q network according to claim 1, wherein, The process of establishing the three-sequence parameter set of the Katyusha algorithm specifically includes: Initialize the three-sequence parameter set, which includes the main parameter, the momentum parameter and the topological parameter; Set the parameter vector for value function update in the main parameter, set the parameter vector for iteration acceleration in the momentum parameter, and set the parameter vector for topological structure constraint in the topological parameter; Input the cross-scale topological embedding set into the topological parameter to establish the initial expression of the topological constraint; Unify the main parameter, the momentum parameter and the topological parameter into the same iteration framework to form the initial state of the three-sequence parameter set. 5.The battery state intelligent regulation method based on deep Q network according to claim 1, wherein, The process of calculating the cross-scale topological consistency loss in the iteration update specifically includes: Determine the short time window persistent interval set based on the short time window topological barcode, and determine the long time window persistent interval set based on the long time window topological barcode; Calculate the length and center position of each persistent interval for the short time window persistent interval set and the long time window persistent interval set respectively, and take the length difference and the center difference as the difference measurement factor; Set the length difference weight and the center difference weight, combine them to form a cross-scale consistency index, and use it to measure the matching relationship between the short time window and the long time window persistent intervals; Limit the one-to-one correspondence condition in the matching relationship, which requires that the short time window persistent interval and the long time window persistent interval can only establish a unique matching relationship, and each persistent interval can only be paired with another persistent interval, and the matching relationship is limited to matching or not matching by an indicator variable; After obtaining the cross-scale consistency index, input the cross-scale consistency index as the topological consistency loss into the topological parameter update process, and use the current topological consistency loss to constrain the topological parameter in the iteration update. 6.The battery state intelligent regulation method based on deep Q network according to claim 1, wherein, The process of introducing the distribution matching regularization in the main parameter update specifically includes: Store the historical spatiotemporal working condition sequence as an experience replay set, and extract the working condition distribution from the experience replay set; Construct the real-time collected spatiotemporal working condition sequence into an online set, and extract the working condition distribution from the online set; Establish a distribution difference function between the experience replay set and the online set, and use the maximum mean difference method to represent the difference between the two types of working condition distributions as a distribution matching regularization term; Input the distribution matching regularization term as a correction signal into the main parameter update process, and superimpose it with the gradient of the main parameter update; In the main parameter update process, combine the correction signal with the original gradient to adjust the main parameter vector, so that the main parameter can reflect the difference between the experience replay set distribution and the online set distribution. 7.The battery state intelligent regulation method based on deep Q network according to claim 1, wherein, The process of generating a limited adversarial disturbance gradient in the momentum parameter update specifically includes: Extract the cell node features from the spatiotemporal working condition sequence, and generate an adversarial disturbance input combined with the cell topological coupling graph; The gradient direction is generated based on a timing difference loss in an adversarial disturbance input, and a disturbance amplitude limit constant is used to limit the disturbance amplitude, to obtain a limited adversarial disturbance gradient; In generating the limited adversarial disturbance gradient, the state variable and the action variable are combined with the immediate reward and the discount factor, and the state-action value is calculated by the deep Q network prediction function and the target network prediction function, so that the limited adversarial disturbance gradient can reflect the update direction under the condition of limited input; The limited adversarial disturbance gradient is superimposed with the momentum parameter original gradient direction to form a momentum update direction; The momentum parameter vector is adjusted according to the momentum update direction to complete the correction of the momentum parameter in the iteration process. 8.The battery state intelligent regulation method based on deep Q network according to claim 1, wherein, The process of completing the three-sequence iterative update and correcting the deep Q network model by using the updated parameter specifically includes: After completing the iterative update of the main parameter, the momentum parameter and the topological parameter, a corrected deep Q network model is formed, which is composed of an input layer, a hidden layer and an output layer. The input layer receives the features extracted from the space-time working condition sequence and the cell topological coupling graph. The hidden layer generates a state feature representation through nonlinear mapping. The output layer calculates the corresponding value of the state and the action. The space-time working condition sequence is input into the corrected deep Q network model to obtain the state-action value corresponding to each candidate action. The action with the maximum value in the state-action values of all candidate actions is selected as the battery control output action. The battery control output action is decomposed into a battery charge and discharge control instruction and a battery thermal management control instruction, and the control instructions are sent to the battery control system for execution. The battery control output action is decomposed into a battery charge and discharge control instruction and a battery thermal management control instruction, and the control instructions are sent to the battery control system for execution.

Images