Lithium battery state of health evaluation method based on compressed multi-head latent attention model

Abstract

This invention provides a lithium battery health status assessment method based on a compressed multi-head latent attention model, belonging to the field of battery state monitoring technology. The method includes: processing and dividing lifecycle test data from lithium battery charge-discharge cycle tests to obtain a multivariate time series of the lithium battery; inputting the multivariate time series into an embedding layer and applying rotational position encoding to the embedding vectors to obtain a feature transformation sequence; injecting the feature transformation sequence into a compressed multi-head latent attention model within the encoder, compressing and aggregating the feature transformation sequence to obtain compressed attention features, and then performing multi-scale feature pooling and concatenation to obtain concatenated features, which are then input into a hybrid expert layer to obtain expert fusion features; and performing linear transformation and nonlinear activation on the expert fusion features to obtain the SOH prediction value. This method effectively solves the challenges faced by existing deep learning models in this field and significantly improves the accuracy, efficiency, and generalization ability of SOH assessment.

Description

Technical Field

[0001] This invention relates to the field of battery status monitoring technology, and more specifically, to a method for assessing the health status of lithium batteries based on a compressed multi-head latent attention model. Background Technology

[0002] In recent years, deep learning models such as Transformer and LSTM have demonstrated advantages in sequence data modeling for SOH (Sequence-Oriented Battery Management). However, Transformer suffers from excessive computational cost and memory consumption. Its core multi-head self-attention mechanism has a computational complexity of O(n²), and during inference, it requires storing a massive "key-value" cache proportional to the sequence length. This results in excessive computational cost and huge memory consumption when processing battery sequence data with hundreds of cycles, making it difficult to achieve real-time online deployment in embedded battery management systems with limited computing resources. The LSTM part is essentially a serial computation, which may become a bottleneck for training and inference speed, especially for long sequences.

[0003] Therefore, existing deep learning models for battery health estimation suffer from the drawback of being unable to balance long sequence modeling capabilities with computational / storage efficiency. Summary of the Invention

[0004] The problem that this invention aims to solve is that existing deep learning models struggle to balance long sequence modeling capabilities with computational / storage efficiency.

[0005] To address the aforementioned problems, in a first aspect, this invention provides a method for assessing the health status of lithium batteries based on a compressed multi-head latent attention model, comprising: The life cycle test data of lithium battery charge-discharge cycle test are processed and divided to obtain the multivariate time series of lithium battery. The multivariate time series is input into the embedding layer to obtain the embedding vector, and rotation position encoding is applied to the embedding vector to obtain the feature transformation sequence; The feature transformation sequence is injected into the compressed multi-head latent attention model in the encoder, and the feature transformation sequence is compressed and aggregated to obtain compressed attention features; Based on the compressed attention features, multi-scale feature pooling is performed, and the pooling results are concatenated to obtain the concatenated features. The spliced ​​features are input into the hybrid expert layer to obtain the expert fusion features; Linear transformation and nonlinear activation are applied to the expert fusion features to obtain the SOH prediction value. During the training process, adversarial training is performed based on the expert fusion features to minimize the SOH prediction loss and maximize the domain classification loss. The parameters of the feature extraction network are adjusted until the first total loss meets the preset convergence condition. The feature extraction network includes an encoder and a hybrid expert layer.

[0006] Secondly, the present invention also provides a lithium battery health status assessment system based on a compressed multi-head latent attention model, comprising: The data processing module is used to process and divide the life cycle test data of the lithium battery charge-discharge cycle test to obtain the multivariate time series of the lithium battery. The embedding transformation module is used to input multivariate time series data into the embedding layer to obtain embedding vectors, and to apply rotation position encoding to the embedding vectors to obtain feature transformation sequences. The encoder module is used to inject the feature transformation sequence into the compressed multi-head latent attention model within the encoder, and to compress and aggregate the feature transformation sequence to obtain compressed attention features. The pooling module is used to perform multi-scale feature pooling based on compressed attention features, and then concatenate the pooling results to obtain concatenated features. The hybrid expert module is used to input the spliced ​​features into the hybrid expert layer to obtain expert fusion features; The prediction module is used to perform linear transformation and nonlinear activation on the expert fusion features to obtain the SOH prediction value. During the training process, adversarial training is performed based on the expert fusion features to minimize the SOH prediction loss and maximize the domain classification loss. The parameters of the feature extraction network are adjusted until the first total loss meets the preset convergence condition. The feature extraction network includes an encoder and a hybrid expert layer.

[0007] This invention provides a method for assessing the health status of lithium batteries based on a compressed multi-head latent attention model. Compared with existing technologies, it has the following advantages: A compressed multi-head latent attention model is introduced, significantly reducing computational complexity and memory requirements by confining attention computation to a lower-dimensional latent space. This enables the model to efficiently process long-sequence data, overcoming the computational efficiency bottleneck of traditional Transformers and providing feasibility for deploying SOH evaluation models in resource-constrained battery management systems. By employing a parallelized attention mechanism and a hybrid expert layer, parallel computing resources are better utilized, resulting in higher efficiency when processing long sequences. Furthermore, multi-scale feature pooling captures information at different temporal granularities from compressed attention features, allowing the model to more comprehensively understand the battery degradation process, rather than relying solely on single-scale features. This multi-scale analysis capability helps improve the accuracy and robustness of SOH prediction. The introduction of a hybrid expert layer allows the model to dynamically activate the most relevant expert network based on the characteristics of the input features. This is equivalent to providing the model with the ability to handle data from different battery aging stages, operating conditions, or battery types, thereby improving the model's flexibility and expressiveness. For example, for different batches or models of batteries, the hybrid expert layer can adaptively select the most suitable expert for processing, which is difficult to achieve in traditional single-model architectures. The introduction of adversarial training is a significant contribution of this method to its generalization ability. By minimizing the SOH prediction loss and maximizing the domain classification loss during training, the feature extraction network is forced to learn robust features that are insensitive to domain changes. This enables the model to maintain high SOH prediction accuracy when faced with unknown battery data from different production batches, models, or operating conditions, effectively solving the domain drift problem commonly encountered in practical applications and significantly improving the model's practical value and reliability. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart illustrating a lithium battery health status assessment method based on a compressed multi-head latent attention model, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a lithium battery health status assessment system based on a compressed multi-head latent attention model, provided in an embodiment of the present invention. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application are described clearly and completely. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0011] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0012] like Figure 1 As shown in the embodiment of this application, a method for assessing the health status of a lithium battery based on a compressed multi-head latent attention model is provided, including: S1: Process and divide the life cycle test data of the lithium battery charge-discharge cycle test to obtain the multivariate time series of the lithium battery.

[0013] Specifically, statistical summaries (mean, standard deviation, minimum, and maximum) of voltage, current, and temperature are extracted from each charge-discharge cycle, along with derived features such as charging time, energy throughput, and voltage curve characteristics. Sequences are constructed using a sliding window of 100 cycles with a step size of 5, where the target value for each sequence is the state of equilibrium (SOH) of the last cycle within the window. All features are standardized with zero mean and unit variance using statistics computed from the training set, and the training, validation, and test sets are partitioned by battery number.

[0014] S2: Input the multivariate time series into the embedding layer to obtain the embedding vector, and apply rotation position encoding to the embedding vector to obtain the feature transformation sequence.

[0015] Specifically, the embedding layer can be a fully connected neural network, whose role is to map the original multivariate time series data into a low-dimensional continuous vector space, thereby capturing the inherent patterns and relationships in the data. For example, for a sequence containing multiple features such as voltage, current, and temperature, the embedding layer can combine these features into a unified embedding vector. Subsequently, in order for the model to understand information at different positions in the sequence and handle long sequence dependencies, rotational position encoding can be applied to these embedding vectors. Rotational position encoding, by introducing rotational transformations into the vectors, allows the model to effectively encode relative positional information without introducing additional absolute positional information, thereby enhancing the model's ability to handle long sequences.

[0016] S3: Inject the feature transformation sequence into the compressed multi-head latent attention model within the encoder, compress and aggregate the feature transformation sequence to obtain compressed attention features.

[0017] Specifically, the model can significantly reduce the computational complexity and memory footprint of the attention mechanism by mapping the original sequence to a lower-dimensional latent space and performing attention computation in that latent space.

[0018] S4: Based on the compressed attention features, perform multi-scale feature pooling and concatenate the pooling results to obtain a concatenated feature. Multi-scale feature pooling aims to capture information at different temporal granularities from the compressed attention features. Concatenating these features extracted at different scales forms a more comprehensive and robust concatenated feature that integrates multi-scale information, helping to more accurately assess battery health.

[0019] S5: Input the spliced ​​features into the hybrid expert layer to obtain the expert fusion features.

[0020] Specifically, the hybrid expert layer consists of a gating network and multiple parallel expert networks. The gating network takes the concatenated features as input, generates the weight distribution of each expert through linear transformation and Softmax activation, and selects the top k experts with the highest scores using a Top-K strategy. Each selected expert network performs an independent nonlinear transformation on the input features. The gating network then weights and sums the outputs of each activated expert according to their corresponding weights to obtain the expert fusion features. During training, a load balancing loss is introduced as an auxiliary constraint to encourage the gating network to distribute the input evenly among the experts, preventing expert network collapse. This load balancing loss is weighted and summed with the SOH prediction loss to form a second total loss, which is then jointly optimized.

[0021] S6: Perform linear transformation and nonlinear activation on the expert fusion features to obtain the SOH prediction value. During the training process, adversarial training is performed based on the expert fusion features to minimize the SOH prediction loss and maximize the domain classification loss. The parameters of the feature extraction network are adjusted until the first total loss meets the preset convergence condition. The feature extraction network includes an encoder and a hybrid expert layer.

[0022] Specifically, the introduction of adversarial training aims to enhance the model's generalization ability, enabling it to maintain high prediction accuracy when faced with lithium battery data from different batches, models, or operating conditions. For example, a domain discriminator network is introduced to distinguish whether input features originate from the source domain (e.g., known battery data used for training) or the target domain (e.g., new battery data encountered in actual deployment). The feature extraction network (including an encoder and a hybrid expert layer) is trained to generate domain-independent features, attempting to deceive the discriminator into failing to distinguish the source domain of the features. Through this adversarial training process, the feature extraction network learns more robust and domain-invariant feature representations, thereby improving the accuracy of SOH predictions on data from unknown domains. The training process continues until the SOH prediction loss and the domain classification loss converge, indicating that the model has learned effective feature representations and prediction capabilities.

[0023] In this optional embodiment, a compressed multi-head latent attention model is introduced. By restricting attention computation to a lower-dimensional latent space, computational complexity and memory requirements are significantly reduced. This enables the model to efficiently process long sequence data, overcoming the computational efficiency bottleneck of traditional Transformers and providing feasibility for deploying SOH evaluation models in resource-constrained battery management systems. By employing a parallelized attention mechanism and a hybrid expert layer, parallel computing resources can be better utilized, resulting in higher efficiency when processing long sequences. Furthermore, multi-scale feature pooling captures information at different temporal granularities from compressed attention features, allowing the model to more comprehensively understand the battery degradation process, rather than relying solely on features at a single scale. This multi-scale analysis capability helps improve the accuracy and robustness of SOH prediction. The introduction of a hybrid expert layer allows the model to dynamically activate the most relevant expert network based on the characteristics of the input features. This effectively provides the model with the ability to handle data from different battery aging stages, operating conditions, or battery types, thereby improving the model's flexibility and expressiveness. For example, the hybrid expert layer can adaptively select the most suitable expert for processing batteries from different batches or models, which is difficult to achieve in traditional single-model architectures. The introduction of adversarial training is a significant contribution of this method to its generalization ability. By minimizing the SOH prediction loss and maximizing the domain classification loss during training, the feature extraction network is forced to learn robust features insensitive to domain changes. This allows the model to maintain high SOH prediction accuracy even when faced with unknown battery data from different production batches, models, or operating conditions, effectively solving the domain drift problem commonly encountered in practical applications and significantly improving the model's practical value and reliability.

[0024] This method combines compressed multi-head latent attention models, multi-scale feature pooling, hybrid expert layers, and adversarial training to provide a lithium battery health status assessment scheme that achieves a good balance between long sequence modeling capability and computational / storage efficiency. It effectively solves the challenges faced by existing deep learning models in this field and significantly improves the accuracy, efficiency, and generalization ability of SOH assessment.

[0025] The following is a detailed description of each step.

[0026] S1: Process and divide the life cycle test data of the lithium battery charge-discharge cycle test to obtain the multivariate time series of the lithium battery.

[0027] Specifically, multiple new batteries of the same model were selected and subjected to full-lifecycle cyclic testing under various operating conditions (different charge / discharge rates and ambient temperatures). Voltage, current, and temperature were recorded at a sampling rate of 1Hz until the capacity decayed to below 80%. Each cycle was divided into three stages: constant current charging, constant voltage charging, and discharging. The mean, standard deviation, minimum, and maximum values ​​of voltage / current / temperature were extracted for each stage, and the charging time, energy throughput, and peak values ​​of the dQ / dV (capacity versus voltage) curves were calculated as health features. A sliding window of length 100 and step size 5 was used to construct a sequence of samples, with the capacity retention rate of the last cycle within the window used as the SOH label. All samples were divided into three mutually exclusive sets according to battery number: 60% of the batteries were used for the training set, responsible for learning model parameters and calculating the mean and standard deviation of all features for subsequent standardization; 20% of the batteries were used for the validation set, monitoring for overfitting during training but not participating in parameter updates; and 20% of the batteries were used for the test set, used only for final performance evaluation and completely invisible until model training was completed. Based on the training set, the multivariate time series is standardized with zero mean and unit variance.

[0028] S2: Input the multivariate time series into the embedding layer to obtain the embedding vector, and apply rotation position encoding to the embedding vector to obtain the feature transformation sequence.

[0029] Specifically, the standardized multivariate time series data is input into the embedding layer and mapped to a high-dimensional model feature space through linear projection, thus achieving a unified dimensional representation of the input features. Subsequently, Rotary Position Embedding (ROPE) is applied to the embedding vectors. This encoding method applies position-related rotation transformations to the query and key vectors, enabling the model to explicitly model the relative positional relationships between sequence elements during attention computation, thereby perceiving the sequence and intervals of charge-discharge cycles. Leveraging ROPE's positional awareness, the model can effectively capture the long-term monotonic decay trend and long-range temporal dependence of battery capacity as the number of cycles increases.

[0030] S3: Inject the feature transformation sequence into the compressed multi-head latent attention model within the encoder, compress and aggregate the feature transformation sequence to obtain compressed attention features.

[0031] Specifically, the key and value vectors in the feature transformation sequence are projected into a shared latent space with significantly reduced dimensionality through a shared low-rank linear transformation, while the query vector retains its original high dimensionality. When calculating attention, the key vectors in the latent space are first reconstructed to the full dimensions of the query vector to ensure the granularity and complete representation of the query-key interaction scoring. Then, the generated attention weights are used to efficiently aggregate the compressed value vectors, and finally, the results are expanded back to the original dimensions. The specific steps are as follows: S31: A fusion projection strategy is used to process the feature transformation sequence to obtain a compressed tensor.

[0032] Asymmetric Query and Key-Value Projection: To improve efficiency, a fused projection strategy is employed instead of independent projection operations. The full-dimensional query vector is projected independently, while the key and value tensors are generated from a shared, compressed projection. To further optimize this process, the compressed matrix and the K / V generation matrix are fused into a single linear layer, reducing the number of matrix multiplication operations.

[0033] The compressed tensor includes fused compressed key-value features, and the query vector is generated by independent query projections.

[0034] The query vector is .

[0035] The fusion compression key value feature is: .

[0036] Where X represents a feature transformation sequence with positional information. Indicates querying the projection matrix. This indicates the dimension of the compressed multi-head latent attention model, where T represents the number of lithium battery charge-discharge cycles. This represents the fused compressed key-value projection matrix. This indicates the compressed dimension.

[0037] S32: Split the compressed tensor to obtain the latent key tensor and the latent value tensor.

[0038] The latent key tensor and potential value tensor They are respectively ,in, This represents the tensor partitioning operation.

[0039] S33: Reconstruct the keys of the latent key tensor to obtain full-dimensional key values.

[0040] To maintain full-rank query-key interactions, we refactor the key to a full dimension.

[0041] The full-dimensional key value is ,in, Represents the key expansion matrix.

[0042] S34: Apply rotational position encoding to both the full-dimensional key value and the query vector simultaneously to obtain the attention weight matrix.

[0043] Before calculating attention, we apply rotational position encoding to both the query vector and the reconstructed key vector.

[0044] The attention weight matrix is: Where softmax is the normalization function, ROPE() represents rotational position encoding, and h represents the number of multi-head attention heads.

[0045] S35: Based on the attention weight matrix and the latent value tensor, obtain the compressed attention features.

[0046] The calculated attention weights are applied to the compressed value tensor, followed by dimensional expansion.

[0047] The compressed attention feature is ,in, This indicates the output projection matrix.

[0048] When processing lithium battery charge-discharge cycle test data, this approach effectively addresses the high computational complexity of traditional attention mechanisms when handling long sequence data. A fusion projection strategy is employed to compress the feature transformation sequence, significantly reducing the dimensionality and computational cost of subsequent attention calculations. By splitting the compressed tensor into latent key tensors and latent value tensors, and reconstructing the latent key tensors, rich semantic information is captured even with low computational cost. Simultaneously, rotational position encoding is applied to the full-dimensional key-value pairs and query vectors, enabling the model to accurately perceive relative positional information within the sequence. This allows for more accurate capture of dependencies between different time steps in the sequence when calculating attention weights. Ultimately, the compressed attention features obtained from the attention weight matrix and latent value tensors not only contain key sequence information but are also obtained under efficient computational conditions. This enables the model to more effectively handle large-scale time-series data when evaluating the health status of lithium batteries, improving the efficiency and accuracy of feature aggregation, thereby enhancing the accuracy of SOH prediction and the training speed of the model.

[0049] S4: Based on the compressed attention features, perform multi-scale feature pooling, and then concatenate the pooling results to obtain the concatenated features.

[0050] S41: Input the compressed attention features into the feedforward neural network in the encoder, and perform nonlinear mapping on the compressed attention features at each time position to obtain deep features.

[0051] S42: Input the compressed attention features and deep features into the residual connection normalization layer in the encoder to obtain a multidimensional feature sequence.

[0052] Specifically, after compressing the multi-head latent attention mechanism, the output undergoes residual connection and layer normalization. Residual connections introduce an identity mapping, creating a direct path for the gradient and effectively mitigating the vanishing gradient problem in deep networks. Layer normalization normalizes the features of each sample, stabilizing the data distribution and smoothing the loss surface, thus jointly improving training stability and accelerating convergence. The processed features are then input into a feedforward neural network, which consists of two linear transformation layers with a non-linear activation function introduced between them, performing independent and deep non-linear mappings on the features at each temporal position. The output of the feedforward network is again residually connected to the input and subjected to layer normalization, further enhancing gradient flow and feature normalization, completing the entire processing flow of an encoder layer and providing a more expressive feature representation for the next layer.

[0053] S43: Perform average pooling, max pooling, and attention pooling on the multidimensional feature sequence, and then concatenate the three pooled features to obtain the concatenated features.

[0054] Specifically, after the multi-layer encoder completes temporal feature extraction, the output remains a multi-dimensional feature sequence with the same length as the input sequence. To obtain a fixed-dimensional global representation suitable for subsequent prediction tasks, this module introduces a multi-scale pooling strategy. This strategy employs three pooling operations in parallel: average pooling to extract the overall evolutionary trend of the sequence, max pooling to capture significant fluctuation features in local regions, and attention pooling to adaptively focus on key temporal positions through a learnable weight mechanism. The outputs of these three operations are concatenated and fused to form a compact feature vector containing both global trends and local details, providing a comprehensive and robust temporal feature representation for subsequent processing.

[0055] Before pooling the compressed attention features, a feedforward neural network is introduced for nonlinear mapping to extract deep features. The original features and deep features are then fused through residual connections and normalization layers, effectively enhancing the expressive power and stability of the features. Based on this, three pooling methods at different scales—average pooling, max pooling, and attention pooling—are employed to comprehensively capture global, local, and key information from the multi-dimensional feature sequence, avoiding information loss that might occur with a single pooling method. Finally, these multi-scale features are concatenated, resulting in richer and more robust concatenated features, thus significantly improving the accuracy and reliability of lithium battery health status assessment.

[0056] S5: Input the spliced ​​features into the hybrid expert layer to obtain the expert fusion features.

[0057] S51: Input the spliced ​​features into the gating network within the hybrid expert layer to obtain a sparse weight distribution.

[0058] The Hybrid Expert Layer (MoE layer) consists of a set of N "expert" networks (simple multilayer perceptrons) and a gating network. The gating network takes pooled features as input and generates a sparse weight distribution for each expert. , Where G(x) is the sparse weight distribution of the gating network output; softmax is the normalization process; TopK represents selecting the top k expert networks with the highest scores; x represents the concatenated features input to the gating network, which are features after pooling; Wg represents the gating weight matrix, which, for a given input, activates only the top k experts with the highest weights.

[0059] S52: Based on the sparse weight distribution, select the top k expert networks with the highest scores for activation, and then sum the outputs of the activated expert networks by weighting to obtain the expert fusion features.

[0060] The expert fusion feature is: , in, Indicates the characteristics of expert fusion; This represents the weights assigned by the gating network to the i-th expert network. This represents the output of the i-th expert network; i∈TopK indicates that only the top k experts selected by the gating network are traversed. This allows the model to learn specialized representations of different subsets of the data, thereby efficiently creating different processing paths for batteries with different characteristics (e.g., different chemical systems, manufacturers, or operating conditions). The final output is a weighted sum of the outputs of the activated experts.

[0061] S6: Perform linear transformation and nonlinear activation on the expert fusion features to obtain the SOH prediction value. During the training process, adversarial training is performed based on the expert fusion features to minimize the SOH prediction loss and maximize the domain classification loss. The parameters of the feature extraction network are adjusted until the first total loss meets the preset convergence condition. The feature extraction network includes an encoder and a hybrid expert layer.

[0062] Specifically, after completing multi-scale pooling and feature transformation of the hybrid expert layer, this module maps high-dimensional features to the final SOH prediction value. The prediction output module consists of a multilayer perceptron, whose input is the fused feature vector output by the hybrid expert layer. Feature compression and abstraction are performed through layer-by-layer linear transformation and nonlinear activation functions. Finally, the output value is constrained between 0 and 1 by the Sigmoid activation function, corresponding to the SOH estimate for the current battery cycle. In terms of model evaluation, in addition to using conventional regression metrics such as mean absolute error and root mean square error to measure prediction accuracy, this scheme also introduces two dedicated metrics: monotonicity violation rate and smoothness index, to verify whether the prediction curve conforms to the physical law of continuous battery capacity decay. This evaluation system ensures that the SOH estimate output by the model is not only numerically accurate but also physically reasonable and credible.

[0063] In an optional embodiment of this application, before using the network model, the network model needs to be trained with a training set. The hybrid expert layer can be trained first, and an auxiliary load balancing loss can be used to encourage the gating network to distribute the input evenly to all experts and prevent expert collapse.

[0064] The hybrid expert layer training process includes: S71: Determine the load balancing loss of the gating network based on the input distribution ratio of multiple expert networks.

[0065] S72: The load balancing loss and the SOH prediction loss are weighted and summed to obtain the second total loss. The parameters of the gating network are adjusted with the goal of minimizing the second total loss.

[0066] Specifically, the load balancing loss is , in, This represents the variance function, used to measure the dispersion of a set of data; The input allocation ratio of the i-th expert network is represented by the percentage of tokens actually processed by the i-th expert in the current batch, and the number of tokens allocated to each expert is divided by the total number of tokens; N represents the number of expert networks in the hybrid expert layer.

[0067] The predicted loss of SOH The mean squared error (MSE) is used for calculation, and its expression is as follows: Where N1 is the sample size. The true SOH value for the i-th sample. This is the predicted value for SOH.

[0068] The purpose of this load balancing loss is to avoid "expert collapse." If the gating network always concentrates the input on a few expert networks, other expert networks will be "idle," and the model's capabilities will not be fully utilized. The smaller the variance, the more evenly the input is distributed among the N experts. Minimizing this loss during training allows the gating network to distribute the input more evenly among all expert networks, ensuring that each expert network learns useful features.

[0069] In the lithium battery health status assessment method, the network model training employs a joint optimization strategy. The total loss is a weighted sum of the SOH prediction loss and adversarial loss or load balancing loss. The load balancing loss encourages the gating network to evenly distribute input to each expert, preventing expert network collapse and ensuring that each expert network in the hybrid expert layer is utilized equitably. This mechanism effectively avoids the problem of some expert networks being overactivated while others are idle, thereby improving the overall robustness and generalization ability of the model. Balanced expert network utilization allows the model to more comprehensively learn and integrate knowledge from different expert networks, thus improving the accuracy and reliability of lithium battery SOH prediction.

[0070] Then, adversarial training is performed. During training, adversarial training is conducted based on expert fusion features, aiming to minimize the SOH prediction loss and maximize the domain classification loss. The parameters of the feature extraction network are adjusted until the first total loss satisfies the preset convergence conditions, including: S81: The expert fusion features are forward propagated through the gradient inversion layer and input into the domain discriminator network to obtain the domain prediction value. The gradient inversion layer is connected between the feature extraction network and the domain discriminator.

[0071] Specifically, an adversarial domain adaptation training strategy is adopted to learn joint features that are both discriminative for SOH prediction and invariant across different domains. Based on the high-dimensional features output by the hybrid expert layer, this module introduces an adversarial domain adaptation mechanism to enhance the model's generalization ability across different operating conditions and battery types. This module consists of a gradient reversal layer and a domain discriminator D. The domain discriminator D is a binary classification neural network. Its input is the feature vector output by the hybrid expert layer, and its output is the predicted probability of the domain to which the feature belongs, used to distinguish whether the feature comes from the source domain or the target domain. It typically consists of several fully connected network layers plus a sigmoid activation function; output values ​​close to 0 indicate the source domain, and close to 1 indicate the target domain. The gradient reversal layer connects the feature extraction network and the domain discriminator. During forward propagation, it does not transform the input features, ensuring normal inference by the discriminator. During backpropagation, this layer multiplies the loss gradient from the domain discriminator by a preset negative constant and then passes it to the front-end feature extraction network, thus causing the parameter update direction of the feature extraction network to be opposite to the optimization objective of the domain discriminator. Through this adversarial training mechanism, the domain discriminator continuously improves its ability to distinguish feature sources, while the feature extraction network gradually learns to generate domain-invariant representations that suppress domain-specific information while preserving key features predicted by SOH. The two achieve a dynamic equilibrium through alternating optimization, ultimately enabling the model to achieve predictive performance on target domain data similar to that of the source domain.

[0072] For a gradient reversal layer, its forward propagation is represented as: Backpropagation is represented as Where GRL(x) represents the output of the gradient inversion layer; λ represents the gradient transformation during backpropagation; λ represents the scaling factor controlling the adversarial adaptive strength; I represents the identity matrix.

[0073] S82: Determine the domain classification loss based on the domain prediction value and the target domain.

[0074] Specifically, the target domain refers to the domain of the data distribution that the model ultimately needs to generalize to in a domain adaptation task. During training, both source domain data and target domain data are typically used simultaneously. Domain classification loss is a metric that measures the difference between the domain predicted by the domain discriminator network and the true target domain.

[0075] The domain classification loss is in, Indicates domain label, The feature representation of the sample, The feature representation of the sample, Indicates the number of source samples. Indicates the target number of samples.

[0076] S83: Backpropagate the domain classification loss, invert the gradient at the gradient inversion layer and pass it to the feature extraction network, and perform adversarial training with the goal of minimizing the SOH prediction loss and maximizing the domain classification loss to determine the first total loss.

[0077] Specifically, the model employs a joint optimization strategy during training: on the one hand, it minimizes the SOH prediction loss to ensure that the feature extraction network retains key information related to battery aging status; on the other hand, it maximizes the neighborhood classification loss through a gradient reversal layer, causing the parameter update direction of the feature extraction network to be opposite to the optimization objective of the neighborhood discriminator. This adversarial training mechanism forms a minimax game process: the neighborhood discriminator continuously improves its ability to distinguish feature sources, while the feature extraction network gradually learns to generate feature representations that can suppress neighborhood-specific information. As iterative training progresses, the two reach equilibrium in the dynamic game, and the feature extraction network eventually produces neighborhood-invariant features that the neighborhood discriminator cannot effectively distinguish, thereby promoting stable prediction performance of the model under different operating conditions and different battery types, and enhancing neighborhood invariance.

[0078] The function for adversarial training is in, This indicates the predicted loss for SOH. Indicates the domain classification loss. The scaling factor representing the control of the adaptive strength of the adversarial system. Represents the learnable parameters of the feature extraction network. This represents the learnable parameters of the network corresponding to the SOH prediction; This represents the learnable parameters of the domain discriminator.

[0079] S84: Adjust the parameters of the feature extraction network according to the first total loss until the first total loss meets the preset convergence condition.

[0080] The first total loss is ,in, This represents the weighting coefficient against losses.

[0081] The smaller the value of the first total loss function, the better; a smaller loss indicates a better model. This function combines the predictive loss and adversarial loss in a weighted manner, optimizing two main objectives: the predictive loss drives the model to extract aging features to ensure estimation accuracy, while the adversarial loss uses gradient inversion to encourage the feature extraction network to learn domain-invariant representations to improve generalization ability. By balancing the weights of both, the model accurately predicts the source domain's state of interest (SOH) while gradually eliminating domain-specificity in the features, thus achieving reliable performance even in unknown target domains.

[0082] Through the above technical solution, this application effectively addresses the problem of insufficient generalization ability of models on different domain data in lithium battery health status assessment. By introducing a gradient reversal layer and a domain discriminator network for adversarial training, the feature extraction network is forced to learn and generate domain-invariant feature representations. This means that even when there are domain differences between the training data and the actual application data, the model can still maintain high SOH prediction accuracy. This mechanism avoids overfitting of the model in a specific training domain, thereby significantly improving the robustness and applicability of the model on unknown or different domain data, making the evaluation results more reliable and stable.

[0083] like Figure 2 As shown in the embodiment of this application, a lithium battery health status assessment system based on a compressed multi-head latent attention model is provided, comprising: The data processing module 10 is used to process and divide the life cycle test data of the lithium battery charge-discharge cycle test to obtain the multivariate time series of the lithium battery.

[0084] Embedding transformation module 2 is used to input the multivariate time series into the embedding layer to obtain the embedding vector, and apply rotation position encoding to the embedding vector to obtain the feature transformation sequence.

[0085] The encoder module 30 is used to inject the feature transformation sequence into the compressed multi-head latent attention model in the encoder, and to compress and aggregate the feature transformation sequence to obtain compressed attention features.

[0086] Pooling module 40 is used to perform multi-scale feature pooling based on compressed attention features, and then concatenate the pooling results to obtain concatenated features.

[0087] The hybrid expert module 50 is used to input the spliced ​​features into the hybrid expert layer to obtain expert fusion features.

[0088] Prediction module 60 is used to perform linear transformation and nonlinear activation on the expert fusion features to obtain the SOH predicted value. The adversarial domain adaptive module 70 is used to perform adversarial training based on expert fusion features during training, with the goal of minimizing the SOH prediction loss and maximizing the domain classification loss. The module adjusts the parameters of the feature extraction network until the first total loss meets the preset convergence condition. The feature extraction network includes an encoder and a hybrid expert layer.

[0089] By systematically combining a compressed multi-head latent attention model with multi-scale feature pooling and a hybrid expert layer, and introducing an adversarial training mechanism, the computational complexity and memory requirements for long sequence processing are effectively reduced, while enhancing the model's generalization ability to different battery types and operating conditions. Specifically, the compressed multi-head latent attention model reduces computational complexity from O(n²) to a linear level, enabling the system to process sequence data with hundreds of cycles within the limited computational resources of an embedded battery management system; multi-scale feature pooling captures multi-stage features of battery degradation through pooling operations of different granularities, improving the comprehensiveness of health status assessment; the hybrid expert layer uses a gating network to dynamically select expert networks, enabling the system to adaptively handle degradation patterns throughout the battery's entire lifecycle; and adversarial training forces the feature extraction network to learn domain-invariant features by adversarially targeting the minimization of SOH prediction loss and the maximization of domain classification loss, effectively mitigating the domain drift problem caused by batch differences in battery deployments. Through the above technical solution, the system significantly reduces the consumption of computing resources while maintaining high-precision SOH prediction, and solves the problem that existing deep learning models cannot achieve both long sequence modeling capability and computing / storage efficiency. This provides a feasible solution for the real-time online deployment of embedded battery management systems.

[0090] The above methods are applied in practice, and specific implementation cases are as follows.

[0091] Example 1: Verifying the effectiveness of the core Compressed Multi-Head Latent Attention (CMLA) mechanism of this invention in improving computational efficiency.

[0092] (a) Data Preparation: The publicly available XJTU battery dataset was selected. Cyclic data from 9 batteries were randomly selected as the training set, 3 as the validation set, and the remaining 3 as the test set. Statistical features were extracted from the voltage, current, and temperature curves of each charge-discharge cycle, and a continuous sequence was constructed in chronological order. (b) Model Construction: The CMLATrans model described in this invention was built. Specific settings were as follows: the model's principal dimension was 192, using 6 attention heads and stacking 3 identical encoder layers. The compression ratio c of the compressed attention module was set to 2, compressing the keys and values ​​to half their original dimensions. In this embodiment, the hybrid expert layer and adversarial training functions were temporarily disabled, and only the basic architecture was tested. (c) Training and Testing: The model was trained using the training set data, aiming to minimize the mean absolute error between the predicted SOH value and the true value. After training, the model performance was evaluated using the test set data. (d) Results: On the test set, as shown in Table 1, the model in this embodiment achieved a mean absolute error of 0.0128, a 12.9% reduction compared to the standard uncompressed attention Transformer. As shown in Table 2, in terms of computational efficiency, thanks to the compressed multi-head latent attention mechanism, the KV cache memory usage is reduced by 4 times, and the value aggregation computation is reduced by 50%. As shown in Table 3, during the actual inference process, the model maintains a similar latency level while significantly reducing memory usage, verifying that the core attention mechanism of this invention achieves an effective balance between accuracy and efficiency in the battery SOH estimation task.

[0093] Table 1 Comparison of Prediction Accuracy Table 2. Memory Efficiency Comparison Table 3. Comparison of computational efficiency The model using CMLA outperformed the standard Transformer in all accuracy metrics, validating the statement that it "achieved similar or even better prediction accuracy." Furthermore, it also showed significant improvements in memory and computational efficiency.

[0094] Example 2: Verifying the effectiveness of the present invention in helping the model adapt to the operating conditions of a new battery.

[0095] (a) Data and Scenario Setting: The XJTU dataset is still used, but a cross-domain learning scenario is created: all battery data tested at room temperature (25°C) is used as the "source domain," and all battery data tested at high temperature (45°C) is used as the "target domain." During training, the model can see all source domain data and their health status labels, but can only see target domain data (without labels). (b) Model Construction: A complete CMLATrans model is built. Based on the configuration in Example 1, all functions are enabled: a hybrid expert layer containing 4 sub-networks is set up, with 2 of them dynamically selected each time; adversarial training strategy is enabled, with its loss weight set to 0.1. (c) Training Strategy: Two-stage adversarial training is adopted. On the one hand, the model learns to accurately predict the SOH of batteries in the source domain; on the other hand, a "domain discriminator" attempts to confuse whether the battery data comes from 25°C or 45°C. This training forces the backbone network of the model to learn to extract general battery aging features that are not affected by temperature. (d) Results: Tested on batteries in the 45°C target region that had never learned labels, as shown in Table 4, the complete model of this invention achieved a root mean square error of 0.018 and a coefficient of determination of 0.915. Compared with the model without region adaptation technology (RMSE 0.034, R² 0.683), the prediction accuracy was improved by 47.1% and the explanatory power by 34.0%. Through t-SNE visualization analysis of the features extracted from the model, it was observed that the battery data feature distributions from the 25°C source region and the 45°C target region were highly mixed. The maximum mean difference decreased from 0.214 to 0.150, and the transfer score increased from 0.346 to 0.940, which intuitively confirmed that the model successfully learned the general aging characteristics unaffected by temperature and had strong cross-condition generalization ability. As shown in Table 5, the ablation experiment shows that the complete CMLATrans structure has the best performance.

[0096] Table 4 Comparison of prediction performance in the target domain (45°C) under different adaptive configurations Table 5. Ablation Experiment Results of Domain Adaptive Module In summary, compared with existing technologies, it has the following beneficial effects: 1. Higher prediction accuracy and more reliable results. (a) Compared with the standard Transformer baseline: In tests on the publicly available XJTU battery dataset, CMLATrans achieved a mean absolute error of 0.0128 and a coefficient of determination of 0.9154. Its MAE is 12.9% lower than the standard Transformer baseline, which directly proves its higher numerical prediction accuracy. (b) Significantly improved physical consistency of predictions: This invention introduces the monotonicity violation rate as an evaluation metric. Experimental results show that CMLATrans significantly reduces the monotonicity violation rate of predicted trajectories from 5.1% in the baseline model to 2.3%. This means that its prediction results more strictly follow the basic physical law of "monotonically decaying" battery capacity, effectively reducing prediction fluctuations that violate common sense, and making the prediction results more reliable in actual BMS decision-making.

[0097] 2. Significantly optimized computational and memory efficiency, making it more suitable for embedded deployment. (a) Reduced theoretical computational complexity: The core CMLA mechanism of this invention reduces the computational complexity of the value aggregation step in the attention module from O(T²d_model) in the standard mechanism to O(T²d_c) by compressing key-value pairs (compression ratio c). For example, theoretically, nearly 2 times the speedup can be achieved when the default compression ratio c=2. (b) Reduced memory footprint: The size of the KV cache is compressed from 2T·d_model in the standard mechanism to T·d_c. Under the default configuration (d_model=192, c=2), a 4-fold reduction in KV cache memory footprint is achieved. This allows the model to run on limited embedded hardware resources when processing long sequences (hundreds of iterations), solving the key bottleneck of standard Transformer's difficulty in deployment due to memory explosion. (c) Excellent trade-off between efficiency and performance: Efficiency analysis experiments show that, with a compression ratio of c=2, the model achieves better accuracy than the standard attention mechanism (MAE 0.0128 vs 0.0147) while bearing a small delay overhead. This indicates that its low-rank bottleneck plays a beneficial role in regularization, achieving a "win-win" situation between efficiency and accuracy.

[0098] 3. Strong cross-domain generalization ability and wider application range. (a) Significantly improved adaptability to new operating conditions: In the cross-temperature domain adaptation experiment from room temperature (25°C, source domain) to high temperature (45°C, target domain), CMLATrans with the complete adaptive module achieved a leap in prediction performance in the target domain: RMSE decreased from 0.034 to 0.018, and R² increased significantly from 0.683 to 0.915. (b) Learned true domain-invariant features: Through t-SNE visualization and quantitative indicators such as maximum mean difference (MMD decreased from 0.214 to 0.150), it was demonstrated that after adversarial training, the features extracted by the model highly overlapped in the distribution of the source and target domain data. This indicates that the model successfully removed temperature-related interference factors and learned the essential laws of battery aging. Therefore, it can "learn by analogy" and maintain high-precision prediction when facing battery types or operating conditions that did not appear in the training, greatly enhancing the practicality and robustness of the model.

[0099] 4. The contributions of the model components are clearly defined, and the architectural advantages have been systematically validated. Quantitative validation through ablation studies: Through systematic ablation experiments (such as removing CMLA, MoE, or adversarial training components respectively), the contribution of each core module was quantitatively demonstrated. Experiments show that the cross-domain adaptive component contributes the most to the overall performance improvement, and its removal results in the most significant performance degradation. This verifies the rationality and necessity of the integrated architecture design of this invention, rather than a simple stacking of modules.

[0100] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0101] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for assessing the health status of lithium batteries based on a compressed multi-head latent attention model, characterized in that, include: The life cycle test data of lithium battery charge-discharge cycle test are processed and divided to obtain the multivariate time series of lithium battery. The multivariate time series is input into the embedding layer to obtain the embedding vector, and rotation position encoding is applied to the embedding vector to obtain the feature transformation sequence; The feature transformation sequence is injected into the compressed multi-head latent attention model in the encoder, and the feature transformation sequence is compressed and aggregated to obtain compressed attention features; Based on the compressed attention features, multi-scale feature pooling is performed, and the pooling results are concatenated to obtain the concatenated features. The spliced ​​features are input into the hybrid expert layer to obtain the expert fusion features; Linear transformation and nonlinear activation are applied to the expert fusion features to obtain the SOH prediction value. During the training process, adversarial training is performed based on the expert fusion features to minimize the SOH prediction loss and maximize the domain classification loss. The parameters of the feature extraction network are adjusted until the first total loss meets the preset convergence condition. The feature extraction network includes an encoder and a hybrid expert layer.

2. The lithium battery health status assessment method based on a compressed multi-head latent attention model as described in claim 1, characterized in that, The step of injecting the feature transformation sequence into the compressed multi-head latent attention model within the encoder, and then compressing and aggregating the feature transformation sequence to obtain compressed attention features includes: A fusion projection strategy is used to process the feature transformation sequence to obtain a compressed tensor; The compressed tensor is split into latent key tensors and latent value tensors; Reconstruct the keys of the latent key tensor to obtain full-dimensional key values; By simultaneously applying rotational position encoding to both the full-dimensional key values ​​and the query vector, an attention weight matrix is ​​obtained. Compressed attention features are obtained based on the attention weight matrix and the latent value tensor.

3. The lithium battery health status assessment method based on a compressed multi-head latent attention model as described in claim 2, characterized in that, The compressed tensor includes fused compressed key-value features, and the query vector is generated by independent query projections; The query vector is ; The fusion compression key value feature is: ; Where X represents a feature transformation sequence with positional information. Indicates querying the projection matrix. This indicates the dimension of the compressed multi-head latent attention model, where T represents the number of lithium battery charge-discharge cycles. This represents the fused compressed key-value projection matrix. Indicates the compressed dimension; The latent key tensor and potential value tensor They are respectively ,in, This represents a tensor partitioning operation; The full-dimensional key value is ,in, Represents the key extension matrix; The attention weight matrix is: Where softmax is the normalization function, ROPE() represents rotation position encoding, and h represents the number of multi-head attention heads; The compressed attention feature is ,in, This indicates the output projection matrix.

4. The lithium battery health status assessment method based on a compressed multi-head latent attention model as described in claim 1, characterized in that, The step of performing multi-scale feature pooling based on compressed attention features and concatenating the pooling results to obtain concatenated features includes: The compressed attention features are input into the feedforward neural network in the encoder, and the compressed attention features at each time position are nonlinearly mapped to obtain deep features. The compressed attention features and deep features are input into the residuals in the encoder and connected to the normalization layer to obtain a multidimensional feature sequence. The multidimensional feature sequence is subjected to average pooling, max pooling and attention pooling, and the three pooled features are concatenated to obtain the concatenated features.

5. The lithium battery health status assessment method based on a compressed multi-head latent attention model as described in claim 1, characterized in that, The step of inputting the spliced ​​features into the hybrid expert layer to obtain expert fusion features includes: The concatenated features are input into a gated network within the hybrid expert layer to obtain a sparse weight distribution. Based on the sparse weight distribution, the top k expert networks with the highest scores are selected for activation, and the outputs of the activated expert networks are weighted and summed to obtain the expert fusion features.

6. The lithium battery health status assessment method based on a compressed multi-head latent attention model as described in claim 5, characterized in that, The sparse weight distribution is as follows: Where G(x) is the sparse weight distribution of the gating network output; softmax is the normalization process; TopK represents selecting the top k expert networks with the highest scores; x represents the concatenated features input to the gating network; Wg represents the gating weight matrix; The expert fusion feature is: ,in, Indicates the characteristics of expert fusion; This represents the weights assigned by the gating network to the i-th expert network. This represents the output of the i-th expert network.

7. The lithium battery health status assessment method based on a compressed multi-head latent attention model as described in claim 1, characterized in that, This also includes training a hybrid expert layer; The hybrid expert layer training process includes: The load balancing loss of the gated network is determined based on the input distribution ratio of multiple expert networks. The load balancing loss and the SOH prediction loss are weighted and summed to obtain the second total loss. The parameters of the gating network are adjusted with the goal of minimizing the second total loss. The load balancing loss is ,in, Represents the variance function. represents the input allocation ratio of the i-th expert network, and N represents the number of expert networks in the hybrid expert layer.

8. The lithium battery health status assessment method based on a compressed multi-head latent attention model as described in claim 1, characterized in that, During the training process, adversarial training is performed based on expert fusion features, aiming to minimize the SOH prediction loss and maximize the domain classification loss. The parameters of the feature extraction network are adjusted until the first total loss satisfies a preset convergence condition, including: The expert fusion features are forward-propagated into the domain discriminator network through the gradient inversion layer to obtain the domain prediction value. The gradient inversion layer connects the feature extraction network and the domain discriminator. Determine the domain classification loss based on the domain prediction value and the target domain; The domain classification loss is backpropagated, and the gradient is inverted in the gradient reversal layer before being passed to the feature extraction network. Adversarial training is performed with the goal of minimizing the SOH prediction loss and maximizing the domain classification loss to determine the first total loss. The parameters of the feature extraction network are adjusted based on the first total loss until the first total loss meets the preset convergence condition.

9. The lithium battery health status assessment method based on a compressed multi-head latent attention model as described in claim 8, characterized in that, The domain classification loss is ； in, Indicates domain label, The feature representation of the sample, The feature representation of the sample, Indicates the number of source samples. Indicates the target sample size; The function for adversarial training is ； in, This indicates the predicted loss for SOH. Indicates the domain classification loss. The scaling factor representing the control of the adaptive strength of the adversarial system. Represents the learnable parameters of the feature extraction network. This represents the learnable parameters of the network corresponding to the SOH prediction; Represents the learnable parameters of the domain discriminator; The first total loss is ； in, This represents the weighting coefficient against losses.

10. A lithium battery health status assessment system based on a compressed multi-head latent attention model, characterized in that, include: The data processing module is used to process and divide the life cycle test data of the lithium battery charge-discharge cycle test to obtain the multivariate time series of the lithium battery. The embedding transformation module is used to input multivariate time series data into the embedding layer to obtain embedding vectors, and to apply rotation position encoding to the embedding vectors to obtain feature transformation sequences. The encoder module is used to inject the feature transformation sequence into the compressed multi-head latent attention model within the encoder, and to compress and aggregate the feature transformation sequence to obtain compressed attention features. The pooling module is used to perform multi-scale feature pooling based on compressed attention features, and then concatenate the pooling results to obtain concatenated features. The hybrid expert module is used to input the spliced ​​features into the hybrid expert layer to obtain expert fusion features; The prediction module is used to perform linear transformation and nonlinear activation on the expert fusion features to obtain the SOH prediction value. During the training process, adversarial training is performed based on the expert fusion features to minimize the SOH prediction loss and maximize the domain classification loss. The parameters of the feature extraction network are adjusted until the first total loss meets the preset convergence condition. The feature extraction network includes an encoder and a hybrid expert layer.

Images