A lithium battery soh estimation method based on knowledge distillation

Abstract

The application discloses a lithium battery SOH estimation method based on knowledge distillation, which comprises the following steps: collecting voltage data in a constant-current charging stage and current data in a constant-voltage charging stage, and forming a data set after data preprocessing; a teacher-student knowledge distillation model is built, which comprises a local feature extraction module, a time sequence degradation memory module and a global interaction module; a distillation method of each module is designed, the local feature extraction module adopts layer attention distillation, the time sequence degradation memory module adopts time sequence attention distillation, and the global interaction module adopts feature alignment distillation; through knowledge distillation, lithium battery degradation features extracted by a teacher model are migrated to a student model to obtain a lightweight SOH estimation model; and the lightweight SOH estimation model is used to estimate SOH online. The application builds a knowledge distillation framework, and migrates knowledge from a complex teacher model to a lightweight student model, so that the number of model parameters is significantly reduced while high-precision estimation of lithium battery SOH is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of lithium batteries, in particular to a lithium battery SOH estimation method based on knowledge distillation. BACKGROUND

[0002] Lithium battery state of health (SOH) estimation is one of the key technologies in battery management systems, which aims to predict the remaining service life of lithium batteries by estimating the SOH of the batteries, so as to optimize the use and maintenance of the batteries, therefore, it is of great significance to comprehensively and accurately estimate the SOH of lithium batteries.

[0003] In recent years, SOH estimation methods based on deep learning have made significant progress, and most scholars use high-precision deep learning models for SOH estimation. However, SOH estimation based on high-precision complex models is limited in real-time systems and embedded devices due to its large number of parameters and high computational requirements, requiring a large amount of computing resources and storage space in actual deployment, which is difficult to meet the real-time and resource efficiency requirements of battery management systems such as electric vehicles.

[0004] Knowledge distillation technology can realize efficient and accurate SOH estimation by migrating the knowledge of complex models to lightweight models. Currently, the research on knowledge distillation in the field of SOH estimation is still in its infancy, and existing technologies generally rely on simple homogenous model distillation, which cannot fully utilize the advantages of different architecture models, especially for complex battery degradation patterns in SOH estimation, traditional methods cannot extract key features through the complementary advantages between heterogeneous models, which is not conducive to the resource-efficient use of edge devices. SUMMARY

[0005] In order to overcome the above-mentioned deficiencies of existing data-driven lithium battery SOH estimation methods, the present application proposes a lithium battery SOH estimation method based on knowledge distillation.

[0006] The technical solution of the present application is as follows:

[0007] S1: lithium battery dataset collection, i.e. constant current-constant voltage charging and constant current discharging are performed on a plurality of lithium batteries of the same model, in each charging and discharging process, voltage data in the constant current charging stage and current data in the constant voltage charging stage are collected and recorded respectively in the constant current, constant voltage charging stage, and the SOH value of the lithium battery in this cycle is calculated after each discharging ends; the lithium battery aging dataset is preprocessed; the above data is divided into a training set, a validation set and a test set, wherein the training set is used for teacher-student model joint optimization, the validation set is used for hyperparameter tuning, and the test set is used for final performance evaluation;

[0008] S2: build a teacher-student knowledge distillation model, the model is composed of the following three functional modules:

[0009] Local feature extraction module: includes a CNN for the teacher model and a lightweight CNN for the student model, wherein the depth of the CNN for the teacher model is 1.5 to 2 times that of the student model, and the number of convolutional kernels in each layer is no less than 1.2 times that of the student model;

[0010] Temporal degradation memory module: includes a Bi-LSTM for the teacher model and a unidirectional LSTM for the student model, wherein the hidden layer dimension of the bidirectional long short-term memory network of the teacher model is 1.5 times that of the student model;

[0011] Global Interaction Module: Composed of the Transformer encoder of the teacher model and the FCNN of the student model. The Transformer encoder includes a multi-head self-attention mechanism, and the FCNN has no more than 3 hidden layers.

[0012] S3: Design distillation methods for each module. Specific steps include: First, using layer attention distillation for the local feature extraction module to align the output features of the convolutional layers of the teacher and student models; second, using temporal attention distillation for the temporal feature extraction module to align the temporal attention distributions of the teacher model's Bi-LSTM and the student model's LSTM; third, using feature alignment distillation for the global feature extraction module to map the self-attention features of the teacher model's Transformer to the fully connected layers of the student model; and finally, designing the overall distillation loss function and iteratively optimizing the student model parameters.

[0013] S4: The teacher model is independently pre-trained using the training set. After the parameters of the teacher model converge, all parameters of the teacher model are frozen, and the student model is trained using the distillation method in S3. Then, the hyperparameters of the student model are adjusted using the validation set to prevent overfitting. Finally, the student model is tested using the test set to verify its performance, and the root mean square error and mean absolute value are used to evaluate the model performance.

[0014] S5: Online estimation of SOH of lithium battery, that is, during the actual charging and discharging process of lithium battery, current data of constant voltage charging stage and voltage data of constant current charging stage are collected. After data preprocessing of the current data of constant voltage charging stage and voltage data of constant current charging stage according to the method of S1, the data is input into the student model trained in S4 to output the estimated value of SOH.

[0015] In this scheme, the distillation method for each module described in step S3 is specifically as follows:

[0016] S301: Design a layer attention distillation method for the local feature extraction module. The layer attention distillation method includes cross-layer attention alignment of the convolutional layer output features of the teacher CNN and the student CNN, generating channel-spatial joint attention weights, and the output of the l-th layer of the teacher CNN. Student features are mapped to the teacher space using learnable convolutional kernels, as shown in the following formula:

[0017]

[0018] in To map student characteristics to the teacher space, W align The convolution kernel is 1×1. For the m-th layer output of the student convolutional neural network, C tea C stu These represent the number of channels in the convolutional neural networks for teachers and students, respectively, and T is the time step.

[0019] By performing global average pooling and global max pooling on the features of the teacher model and student model, the channel weight vector of the layer attention weight is generated, as shown in the following formula:

[0020]

[0021] in These are the global average pooling weight vector and the global max pooling weight vector representing the teacher features, respectively. These are the global average pooling weight vector and the global max pooling weight vector for student features, respectively. GAP and GMP are the global average pooling result and the global max pooling result, respectively. σ is the Sigmoid activation function.

[0022] In the local feature extraction module, the attention map features are determined by the channel weight vector, as shown in the following formula:

[0023]

[0024] Where M tea M stu These are the attention feature maps after alignment of the teacher model and the student model, respectively.

[0025] In the local feature extraction module, the distillation loss function includes a feature map alignment term, as shown in the following equation:

[0026]

[0027] in, M represents the distillation loss of the local feature extraction module. tea [i]、M stu[i] represents the ith layer of the attention feature map after the teacher model and student model are aligned; n represents the total number of layers of the attention feature map after the teacher model and student model are aligned.

[0028] S302: Design a temporal attention distillation method, wherein the temporal attention distillation method includes encoding the temporal features of the teacher model and the student model, and retrieving the forward hidden state of the teacher model. and backward hidden state The splicing is in a two-way hidden state, as shown in the following formula:

[0029]

[0030] The temporal attention weights of the teacher's Bi-LSTM are calculated using only the forward hidden states, while the temporal attention weights of the student's LSTM are based on the one-way hidden states, as shown in the following formula:

[0031]

[0032] in These are the temporal attention weights for the teacher model and the student model, respectively. These represent the hidden states of the teacher model Bi-LSTM at time step t and the student model LSTM, respectively. tea v stu , respectively, are the learnable context vectors of the teacher model and the student model; T is the total time steps of the hidden states of the teacher model Bi-LSTM and the student model LSTM.

[0033] In the temporal degradation memory module, the distillation loss function includes teacher-weighted features and student-weighted features, as shown in the following formula:

[0034]

[0035] in W represents the distillation loss of the time-series feature extraction module. c The teacher bidirectional feature compression matrix ||·||2 represents the L2 norm normalization of the temporal features of the teacher and student models;

[0036] S303: Design a global interaction module feature alignment distillation method. This method extracts the self-attention features of the teacher model. Assume the teacher model contains L layers of Transformer structures, each with H attention head matrices. Taking the l-th layer as an example, each attention head matrix... The statistics of the attention head matrix are calculated along the sequence dimension, and the statistics are concatenated to obtain the global feature vector of the l-th layer, as shown in the following formula:

[0037]

[0038] in This is the query, key, and value matrix for the h-th head.

[0039]

[0040] in These are the mean, variance, and entropy of the l-th layer in the h-th head attention matrix, respectively.

[0041]

[0042] in This represents the global feature vector of the l-th layer of the teacher model.

[0043] The method takes the output of the m-th layer of the student model FCNN and maps it to the teacher model for feature alignment using a projection matrix, as shown in the following formula:

[0044]

[0045] in The feature matrix projected from the student model onto the teacher model. The hidden output of the m-th layer of the student model. The projection matrix of the student model;

[0046] In the global interaction module, the distillation loss function is composed of minimizing the cosine distance, and the feature alignment loss is calculated as follows:

[0047]

[0048] in This is the distillation loss function for the global feature extraction module. These are the normalized matrices of the l-th layer feature matrix of the teacher model and the m-th layer projected feature matrix of the student model, respectively.

[0049] S304: Design the complete distillation loss function for the teacher-student model, as shown in the following formula:

[0050]

[0051] in Let λ1, λ2, and λ3 be the total distillation loss function, respectively. The weight parameters.

[0052] In this scheme, step S4, training and testing the teacher model and the student model, specifically includes:

[0053] S401: Train the teacher model using the training set, and freeze the teacher model's parameters after the model parameters converge.

[0054] S402: Align the convolutional layer outputs of the teacher model and the student model using the layer attention distillation method in step S3; align the LSTM hidden state distributions of the teacher model and the student model using the temporal attention distillation method; map the teacher model Transformer self-attention features to the fully connected layer of the student model FCNN using the feature alignment distillation method in step S3, and perform joint training by combining it with the SOH prediction task loss;

[0055] S403: Use the validation set to adjust the hyperparameters of the student model to prevent overfitting;

[0056] S404: Use the test set to calculate the root mean square error and mean absolute error of the SOH estimate of the student model to verify the performance of the student model;

[0057] In this scheme, during the actual operation described in step S5, online SOH estimation is performed using the SOH estimation model, specifically including:

[0058] S501: Deploy the trained student model to the battery management system;

[0059] S502: The lithium battery management system collects voltage data during the constant current charging stage and current data during the constant voltage charging stage of the lithium battery. After data preprocessing, it uses the SOH estimation model to perform online SOH estimation.

[0060] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0061] This invention proposes a knowledge distillation-based SOH estimation method for lithium batteries. It utilizes a high-precision deep learning model as the teacher model and a lightweight deep learning model as the student model, fully leveraging the complex representational capabilities of the teacher model and the lightweight advantage of the student model. While maintaining high-precision SOH estimation for lithium batteries, it significantly improves the adaptability of the SOH estimation method on embedded devices with lower computing power. Simultaneously, by designing a distillation method between heterogeneous models, the student model can utilize the advantages of different architectures to extract features from battery aging data from three dimensions: local features, temporal degradation memory, and global interactions. This extracts multimodal features related to battery SOH, improving SOH estimation accuracy. Both of these advantages contribute to improving the performance of lithium battery SOH estimation. Attached Figure Description

[0062] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This invention provides a knowledge-based distillation-based method for estimating the state of oxygen (SOH) of a lithium battery.

[0064] Figure 2 Voltage data curves for the constant current charging stage of lithium batteries.

[0065] Figure 3 The current data curve for the constant voltage charging stage of the lithium battery. Detailed Implementation

[0066] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0067] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0068] For easier understanding, please refer to Figure 1 This invention provides an embodiment of a knowledge distillation-based method for estimating the state of oxygen (SOH) of lithium batteries, comprising:

[0069] S1: Lithium battery dataset collection, which involves constant current-constant voltage charging and constant current discharging of multiple lithium batteries of the same model. During each charge and discharge process, voltage data during the constant current charging stage and current data during the constant voltage charging stage are collected and recorded respectively. After each discharge, the SOH value of the lithium battery in this cycle is calculated. The lithium battery aging dataset is preprocessed. The above data is divided into training set, validation set and test set. The training set is used for joint optimization of teacher-student model, the validation set is used for hyperparameter tuning, and the test set is used for final performance evaluation.

[0070] S2: Construct a teacher-student knowledge distillation model, which consists of the following three functional modules:

[0071] Local feature extraction module: includes a convolutional neural network (CNN) for the teacher model and a lightweight CNN for the student model, wherein the depth of the CNN for the teacher model is 1.5 to 2 times that of the student model, and the number of convolutional kernels in each layer is no less than 1.2 times that of the student model;

[0072] Temporal degradation memory module: includes a Bi-LSTM for the teacher model and a unidirectional LSTM for the student model, wherein the hidden layer dimension of the bidirectional long short-term memory network of the teacher model is 1.5 times that of the student model;

[0073] Global Interaction Module: Composed of the Transformer encoder of the teacher model and the FCNN of the student model. The Transformer encoder includes a multi-head self-attention mechanism, and the FCNN has no more than 3 hidden layers.

[0074] S3: Design distillation methods for each module. Specific steps include: First, using layer attention distillation for the local feature extraction module to align the output features of the convolutional layers of the teacher and student models; second, using temporal attention distillation for the temporal feature extraction module to align the temporal attention distributions of the teacher model's Bi-LSTM and the student model's LSTM; third, using feature alignment distillation for the global feature extraction module to map the self-attention features of the teacher model's Transformer to the fully connected layers of the student model; and finally, designing the overall distillation loss function and iteratively optimizing the student model parameters.

[0075] S4: The teacher model is independently pre-trained using the training set. After the parameters of the teacher model converge, all parameters of the teacher model are frozen, and the student model is trained using the distillation method in S3. Then, the hyperparameters of the student model are adjusted using the validation set to prevent overfitting. Finally, the student model is tested using the test set to verify its performance, and the root mean square error and mean absolute value are used to evaluate the model performance.

[0076] S5: Online estimation of SOH of lithium battery, that is, during the actual charging and discharging process of lithium battery, current data of constant voltage charging stage and voltage data of constant current charging stage are collected. After data preprocessing of the current data of constant voltage charging stage and voltage data of constant current charging stage according to the method of S1, the data is input into the student model trained in S4 to output the estimated value of SOH.

[0077] In this scheme, the distillation method for each module described in step S3 is specifically as follows:

[0078] S301: Design a layer attention distillation method for the local feature extraction module. The layer attention distillation method includes cross-layer attention alignment of the convolutional layer output features of the teacher CNN and the student CNN, generating channel-spatial joint attention weights, and the output of the l-th layer of the teacher CNN. Student features are mapped to the teacher space using learnable convolutional kernels, as shown in the following formula:

[0079]

[0080] in To map student characteristics to the teacher space, W align The convolution kernel is 1×1. For the m-th layer output of the student convolutional neural network, C tea C stu These represent the number of channels in the convolutional neural networks for teachers and students, respectively, and T is the time step.

[0081] By performing global average pooling and global max pooling on the features of the teacher model and student model, the channel weight vector of the layer attention weight is generated, as shown in the following formula:

[0082]

[0083] in These are the global average pooling weight vector and the global max pooling weight vector representing the teacher features, respectively. These are the global average pooling weight vector and the global max pooling weight vector for student features, respectively. GAP and GMP are the global average pooling result and the global max pooling result, respectively. σ is the Sigmoid activation function.

[0084] In the local feature extraction module, the attention map features are determined by the channel weight vector, as shown in the following formula:

[0085]

[0086] Where M tea M stu These are the attention feature maps after alignment of the teacher model and the student model, respectively.

[0087] In the local feature extraction module, the distillation loss function includes a feature map alignment term, as shown in the following equation:

[0088]

[0089] in, M represents the distillation loss of the local feature extraction module. tea [i]、M stu[i] represents the ith layer of the attention feature map after the teacher model and student model are aligned; n represents the total number of layers of the attention feature map after the teacher model and student model are aligned.

[0090] S302: Design a temporal attention distillation method, wherein the temporal attention distillation method includes encoding the temporal features of the teacher model and the student model, and retrieving the forward hidden state of the teacher model. and backward hidden state The splicing is in a two-way hidden state, as shown in the following formula:

[0091]

[0092] The temporal attention weights of the teacher's Bi-LSTM are calculated using only the forward hidden states, while the temporal attention weights of the student's LSTM are based on the one-way hidden states, as shown in the following formula:

[0093]

[0094] in Temporal attention weights for the teacher model and the student model, respectively. These represent the hidden states of the teacher model Bi-LSTM at time step t and the student model LSTM, respectively. tea v stu , respectively, are the learnable context vectors of the teacher model and the student model; T is the total time steps of the hidden states of the teacher model Bi-LSTM and the student model LSTM.

[0095] In the temporal degradation memory module, the distillation loss function includes teacher-weighted features and student-weighted features, as shown in the following formula:

[0096]

[0097] in W represents the distillation loss of the time-series feature extraction module. c The teacher bidirectional feature compression matrix ||·||2 represents the L2 norm normalization of the temporal features of the teacher and student models;

[0098] S303: Design a global interaction module feature alignment distillation method. This method extracts the self-attention features of the teacher model. Assume the teacher model contains L layers of Transformer structures, each with H attention head matrices. Taking the l-th layer as an example, each attention head matrix... The statistics of the attention head matrix are calculated along the sequence dimension, and the statistics are concatenated to obtain the global feature vector of the l-th layer, as shown in the following formula:

[0099]

[0100] in This is the query, key, and value matrix for the h-th head.

[0101]

[0102] in These are the mean, variance, and entropy of the l-th layer in the h-th head attention matrix, respectively.

[0103]

[0104] in This represents the global feature vector of the l-th layer of the teacher model.

[0105] The method takes the output of the m-th layer of the student model FCNN and maps it to the teacher model for feature alignment using a projection matrix, as shown in the following formula:

[0106]

[0107] in The feature matrix projected from the student model onto the teacher model. The hidden output of the m-th layer of the student model. The projection matrix of the student model;

[0108] In the global interaction module, the distillation loss function is composed of minimizing the cosine distance, and the feature alignment loss is calculated as follows:

[0109]

[0110] in This is the distillation loss function for the global feature extraction module. These are the normalized matrices of the l-th layer feature matrix of the teacher model and the m-th layer projected feature matrix of the student model, respectively.

[0111] S304: Design the complete distillation loss function for the teacher-student model, as shown in the following formula:

[0112]

[0113] in Let λ1, λ2, and λ3 be the total distillation loss function, respectively. Weight parameters;

[0114] In this scheme, step S4, training and testing the teacher model and the student model, specifically includes:

[0115] S401: Train the teacher model using the training set, and freeze the teacher model's parameters after the model parameters converge.

[0116] S402: Align the convolutional layer outputs of the teacher model and the student model using the layer attention distillation method in step S3; align the LSTM hidden state distributions of the teacher model and the student model using the temporal attention distillation method; map the teacher model Transformer self-attention features to the fully connected layer of the student model FCNN using the feature alignment distillation method in step S3, and perform joint training by combining it with the SOH prediction task loss;

[0117] S403: Use the validation set to adjust the hyperparameters of the student model to prevent overfitting;

[0118] S404: Use the test set to calculate the root mean square error and mean absolute error of the SOH estimate of the student model to verify the performance of the student model;

[0119] In this scheme, during the actual operation described in step S5, online SOH estimation is performed using the SOH estimation model, specifically including:

[0120] S501: Deploy the trained student model to the battery management system;

[0121] S502: The lithium battery management system collects voltage data during the constant current charging stage and current data during the constant voltage charging stage of the lithium battery. After data preprocessing, it uses the SOH estimation model to perform online SOH estimation.

[0122] Reference to the voltage data curve of the lithium battery constant current charging stage Figure 2 The graph shows the time on the horizontal axis (in seconds) and the current on the vertical axis (in amperes). As time increases, the charging current gradually decreases until it stabilizes when the battery is nearly fully charged.

[0123] Reference to the current data curve of lithium battery constant voltage charging stage Figure 3 The horizontal axis represents time in seconds, and the vertical axis represents voltage in volts. As time increases, the charging voltage gradually increases until it stabilizes when the battery is nearly fully charged.

Claims

1. A method for estimating the state of matter (SOH) of a lithium battery based on knowledge distillation, characterized in that, The specific steps are as follows: S1: Lithium battery dataset collection, which involves constant current-constant voltage charging and constant current discharging of multiple lithium batteries of the same model. During each charge and discharge process, voltage data during the constant current charging stage and current data during the constant voltage charging stage are collected and recorded respectively. After each discharge, the SOH value of the lithium battery in this cycle is calculated. The lithium battery aging dataset is preprocessed. The above data is divided into training set, validation set and test set. The training set is used for joint optimization of teacher-student model, the validation set is used for hyperparameter tuning, and the test set is used for final performance evaluation. S2: Construct a teacher-student knowledge distillation model, which consists of the following three functional modules: Local feature extraction module: includes a convolutional neural network (CNN) for the teacher model and a lightweight CNN for the student model, wherein the depth of the CNN for the teacher model is 1.5 to 2 times that of the student model, and the number of convolutional kernels in each layer is no less than 1.2 times that of the student model; Temporal degradation memory module: includes a bidirectional long short-term memory network (Bi-LSTM) for the teacher model and a unidirectional long short-term memory network (LSTM) for the student model, wherein the hidden layer dimension of the bidirectional long short-term memory network for the teacher model is 1.5 times that of the student model; Global interaction module: It consists of a Transformer encoder of the teacher model and a fully connected neural network (FCNN) of the student model. The Transformer encoder includes a multi-head self-attention mechanism, and the FCNN has no more than 3 hidden layers. S3: Design distillation methods for each module. Specific steps include: First, using layer attention distillation for the local feature extraction module to align the output features of the convolutional layers of the teacher and student models; second, using temporal attention distillation for the temporal feature extraction module to align the temporal attention distributions of the teacher model's Bi-LSTM and the student model's LSTM; third, using feature alignment distillation for the global feature extraction module to map the self-attention features of the teacher model's Transformer to the fully connected layers of the student model; and finally, designing the overall distillation loss function and iteratively optimizing the student model parameters. S4: The teacher model is independently pre-trained using the training set. After the parameters of the teacher model converge, all parameters of the teacher model are frozen, and the student model is trained using the distillation method in S3. Then, the hyperparameters of the student model are adjusted using the validation set to prevent overfitting. Finally, the student model is tested using the test set to verify its performance, and the root mean square error and mean absolute value are used to evaluate the model performance. S5: Online estimation of SOH of lithium battery, that is, during the actual charging and discharging process of lithium battery, current data of constant voltage charging stage and voltage data of constant current charging stage are collected. After data preprocessing of the current data of constant voltage charging stage and voltage data of constant current charging stage according to the method of S1, the data is input into the student model trained in S4 to output the estimated value of SOH.

2. The lithium battery SOH estimation method based on knowledge distillation according to claim 1, characterized in that, The distillation method between the modules described in step S3 is as follows: S301: Design a layer attention distillation method for the local feature extraction module. The layer attention distillation method includes cross-layer attention alignment of the convolutional layer output features of the teacher CNN and the student CNN, generating channel-spatial joint attention weights. Layer output The student features are mapped to the teacher space using learnable convolutional kernels, as shown in the following formula: in To map student characteristics to the teacher space, The convolution kernel is 1×1. The output of the m-th layer of the student convolutional neural network. , These represent the number of channels in the teacher's and student's convolutional neural networks, respectively, and T is the time step. By performing global average pooling and global max pooling on the features of the teacher model and student model, the channel weight vector of the layer attention weight is generated, as shown in the following formula: in , These are the global average pooling weight vector and the global max pooling weight vector representing the teacher features, respectively. , These are the global average pooling weight vector and the global max pooling weight vector for student features, respectively. , These are the results of global average pooling and global max pooling, respectively. Use the Sigmoid activation function; In the local feature extraction module, the attention map features are determined by the channel weight vector, as shown in the following formula: in , These are the attention feature maps after alignment of the teacher model and the student model, respectively. In the local feature extraction module, the distillation loss function includes a feature map alignment term, as shown in the following equation: in, Distillation loss of the local feature extraction module , , i, represent the attention feature maps of the i-th layer after alignment of the teacher model and the student model; n represents the total number of layers of the attention feature maps after alignment of the teacher model and the student model. S302: Design a temporal attention distillation method, wherein the temporal attention distillation method includes encoding the temporal features of the teacher model and the student model, and retrieving the forward hidden state of the teacher model. and backward hidden state The splicing is in a two-way hidden state, as shown in the following formula: The temporal attention weights of the teacher's Bi-LSTM are calculated using only the forward hidden states, while the temporal attention weights of the student's LSTM are based on the one-way hidden states, as shown in the following formula: in , These are the temporal attention weights for the teacher model and the student model, respectively. , These represent the hidden states of the teacher model Bi-LSTM and the student model LSTM at time step t, respectively. , , respectively, are the learnable context vectors of the teacher model and the student model; T is the total time steps of the hidden states of the teacher model Bi-LSTM and the student model LSTM. In the temporal degradation memory module, the distillation loss function includes teacher-weighted features and student-weighted features, as shown in the following formula: in The distillation loss of the time-series feature extraction module. Teacher bidirectional feature compression matrix L2 norm normalization for the temporal features of the teacher and student models; S303: Design a global interaction module feature alignment distillation method. This method extracts the self-attention features of the teacher model. Assume the teacher model contains L layers of Transformer structures, each with H attention head matrices. Let the first layer be the self-attention feature of the teacher model. Taking a layer as an example, take the attention head matrix for each layer. Calculate the statistics of the attention head matrix along the sequence dimension, and concatenate the statistics to obtain the first... The global feature vector of the layer is as follows: in , , This is the query, key, and value matrix for the h-th head; in , , The h-th head attention matrix is ​​respectively the h-th head attention matrix. Mean, variance, and entropy of the layer in For the teacher model The global feature vector of the layer; The method takes the output of the m-th layer of the student model FCNN and maps it to the teacher model for feature alignment using a projection matrix, as shown in the following formula: in The feature matrix projected from the student model onto the teacher model. The hidden output of the m-th layer of the student model. The projection matrix of the student model; In the global interaction module, the distillation loss function is composed of minimizing the cosine distance, and the feature alignment loss is calculated as follows: in This is the distillation loss function for the global feature extraction module. The teacher model is the first The normalized matrix of the layer feature matrix and the projection feature matrix of the m-th layer of the student model; S304: Design the complete distillation loss function for the teacher-student model, as shown in the following formula: in Let the total distillation loss function be . , , They are respectively , , The weight parameters.

3. The lithium battery SOH estimation method based on knowledge distillation according to claim 1, characterized in that, The specific steps for training the teacher model and student model in step S4 are as follows: S401: Train the teacher model using the training set, and freeze the teacher model's parameters after the model parameters converge. S402: Align the convolutional layer outputs of the teacher model and the student model using the layer attention distillation method in step S3; align the LSTM hidden state distributions of the teacher model and the student model using the temporal attention distillation method; map the teacher model Transformer self-attention features to the fully connected layer of the student model FCNN using the feature alignment distillation method in step S3, and perform joint training by combining it with the SOH prediction task loss; S403: Use the validation set to adjust the hyperparameters of the student model to prevent overfitting; S404: Use the test set to calculate the root mean square error and mean absolute error of the student model's SOH estimate to verify the student model's performance.

4. The lithium battery SOH estimation method based on knowledge distillation according to claim 1, characterized in that, In the actual operation described in step S5, the online SOH estimation is performed using the lightweight SOH estimation model. The specific steps are as follows: S501: Deploy the trained student model to the battery management system; S502: The lithium battery management system collects voltage data during the constant current charging stage and current data during the constant voltage charging stage of the lithium battery. After data preprocessing, it uses the SOH estimation model to perform online SOH estimation.

Images