Lithium battery SOH estimation method based on knowledge distillation

Abstract

The invention discloses a lithium battery SOH estimation method based on knowledge distillation, and the method comprises the steps: collecting voltage data in a constant-current charging stage and current data in a constant-voltage charging stage, and carrying out the preprocessing of the data, and then forming a data set; building a teacher-student knowledge distillation model, wherein the model comprises a local feature extraction module, a time sequence degradation memory module and a global interaction module; a distillation method of each module is designed, a local feature extraction module adopts layer attention distillation, a time sequence degeneration memory module adopts time sequence attention distillation, and a global interaction module adopts feature alignment distillation; through knowledge distillation, lithium battery degradation characteristics extracted by the teacher model are migrated to the student model, and a lightweight SOH estimation model is obtained; and estimating SOH on line by using the lightweight SOH estimation model. According to the method, a knowledge distillation framework is built, knowledge is transferred from a complex teacher model to a light student model, and the number of model parameters is remarkably reduced while high-precision estimation of the SOH of the lithium battery is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of lithium batteries, and in particular to a method for estimating the state of health (SOH) of lithium batteries based on knowledge distillation. Background Art

[0002] The estimation of the state of health (SOH) of lithium batteries is one of the key technologies in the battery management system. Its purpose is to predict the remaining service life of lithium batteries by estimating the SOH of the batteries, so as to optimize the use and maintenance of the batteries. Therefore, it is of great significance to comprehensively and accurately estimate the SOH of lithium batteries.

[0003] In recent years, significant progress has been made in the SOH estimation method based on deep learning. Most scholars use high-precision deep learning models for SOH estimation. However, for the SOH estimation based on high-precision complex models, its large number of parameters and high computational requirements limit its application in real-time systems and embedded devices. A large amount of computational resources and storage space are required in actual deployment, which is difficult to meet the requirements of real-time performance and resource efficiency of battery management systems such as electric vehicles.

[0004] Knowledge distillation technology can transfer the knowledge of complex models to lightweight models to achieve efficient and accurate SOH estimation. Currently, the research on knowledge distillation in the field of SOH estimation is still in its infancy. The existing technologies generally rely on simple homogeneous model distillation and cannot make full use of the advantages of different architecture models. Especially for the complex battery degradation modes in SOH estimation, traditional methods cannot extract key features through the complementary advantages between heterogeneous models, which is not conducive to the efficient use of resources of edge devices. Summary of the Invention

[0005] In order to overcome the above deficiencies of the existing data-driven methods for estimating the SOH of lithium batteries, the present invention proposes a method for estimating the SOH of lithium batteries based on knowledge distillation.

[0006] The technical solution of the present invention is as follows:

[0007] S1: Collecting a lithium battery dataset, that is, performing constant current-constant voltage charging and constant current discharging on multiple lithium batteries of the same model. During each charge and discharge process, the voltage data in the constant current charging stage and the current data in the constant voltage charging stage are respectively collected and recorded. After each discharge ends, calculate the SOH value of the lithium battery in this cycle; preprocess the lithium battery aging dataset; divide the above data into a training set, a validation set and a test set, where the training set is used for the joint optimization of the teacher-student model, the validation set is used for hyperparameter tuning, and the test set is used for the final performance evaluation;

[0008] S2: Building a teacher-student knowledge distillation model, which is composed of the following three functional modules:

[0009] Local feature extraction module: It includes the CNN of the teacher model and the lightweight CNN of the student model. The depth of the CNN of the teacher model is 1.5 - 2 times that of the student model, and the number of convolutional kernels in each layer is not less than 1.2 times that of the student model.

[0010] Temporal degradation memory module: It includes the Bi-LSTM of the teacher model and the unidirectional LSTM of the student model. The hidden layer dimension of the bidirectional long short-term memory network of the teacher model is 1.5 times that of the student model.

[0011] Global interaction module: It is composed of the Transformer encoder of the teacher model and the FCNN of the student model. The Transformer encoder contains a multi-head self-attention mechanism, and the number of hidden layers of the FCNN does not exceed 3 layers.

[0012] S3: Design the distillation method for each module. The specific steps are as follows: First, use the layer attention distillation method for the local feature extraction module to align the output features of the convolutional layers of the teacher model and the student model; use the temporal attention distillation method for the temporal feature extraction module to align the temporal attention distributions of the Bi-LSTM of the teacher model and the LSTM of the student model through the temporal attention distillation module; use the feature alignment distillation method for the global feature extraction module to map the self-attention features of the Transformer of the teacher model to the fully connected layer of the student model; design the total distillation loss function and iteratively optimize the parameters of the student model.

[0013] S4: Independently pre-train the teacher model using the training set. After the parameters of the teacher model converge, freeze all the parameters of the teacher model, and perform knowledge transfer training on the student model based on the distillation method in S3; then use the validation set to adjust the hyperparameters of the student model to prevent overfitting; finally, use the test set to verify the performance of the trained student model, and use the root mean square error and the mean absolute value to evaluate the model performance.

[0014] S5: Online estimate the SOH of the lithium battery, that is, during the actual charge and discharge process of the lithium battery, collect the current data in the constant voltage charging stage and the voltage data in the constant current charging stage during the charging process. After preprocessing the current data in the constant voltage charging stage and the voltage data in the constant current charging stage according to the method in S1, input them into the trained student model described in S4 to make it output the SOH estimation value.

[0015] In this solution, the distillation method for designing each module described in step S3 is specifically as follows:

[0016] S301: Design the layer attention distillation method for the local feature extraction module. The layer attention distillation method includes cross-layer attention alignment of the output features of the convolutional layers of the teacher CNN and the student CNN to generate channel-space joint attention weights, and the output of the l-th layer of the teacher CNN Map the student features to the teacher space through a learnable convolutional kernel, specifically as follows:

[0017]

[0018] Where is the student feature mapped to the teacher space, and W align is a 1×1 convolutional kernel, is the output of the m-th layer of the student convolutional neural network, and C tea , C stu are the number of channels of the teacher and student convolutional neural networks respectively, and T is the time step;

[0019] Generate the channel weight vectors of the layer attention weights by performing global average pooling and global max pooling on the features of the teacher model and the student model, specifically as follows:

[0020]

[0021] Where are the global average pooling weight vector and the global max pooling weight vector of the teacher features respectively, are the global average pooling weight vector and the global max pooling weight vector of the student features respectively, GAP and GMP are the global average pooling result and the global max pooling result respectively, and σ is the Sigmoid activation function;

[0022] In the local feature extraction module, the attention map features are determined by the channel weight vectors, specifically as follows:

[0023]

[0024] Where M tea , M stu are the attention feature maps after alignment of the teacher model and the student model respectively;

[0025] In the local feature extraction module, the distillation loss function includes a feature map alignment term, specifically as follows:

[0026]

[0027] Where, is the distillation loss of the local feature extraction module, and M tea [i], M stu[i] are the i-th layer attention feature maps after alignment of the teacher model and the student model respectively; n is the total number of layers of the attention feature maps after alignment of the teacher model and the student model.

[0028] S302: Design a temporal attention distillation method, where the temporal attention distillation method includes encoding the temporal features of the teacher model and the student model, and concatenating the forward hidden state and the backward hidden state of the teacher model into a bidirectional hidden state, as shown in the following formula:

[0029]

[0030] The temporal attention weights of the teacher Bi-LSTM are calculated using only the forward hidden state, and the temporal attention weights of the student LSTM are based on the unidirectional hidden state, as shown in the following formula:

[0031]

[0032] where are the temporal attention weights of the teacher model and the student model respectively, are the hidden states of the teacher model Bi-LSTM at time step t and the hidden states of the student model LSTM respectively, v tea , v stu are the learnable context vectors of the teacher model and the student model respectively; T is the total number of time steps of the hidden states of the teacher model Bi-LSTM and the hidden states of the student model LSTM;

[0033] In the temporal degradation memory module, the distillation loss function includes the teacher weighted features and the student weighted features, as shown in the following formula:

[0034]

[0035] where is the distillation loss of the temporal feature extraction module, W c is the teacher bidirectional feature compression matrix ||·|| 2 is the L2 norm normalization of the temporal features of the teacher model and the student model;

[0036] S303: Design a global interaction module feature alignment distillation method, where the global interaction module feature alignment distillation method extracts the self-attention features of the teacher model. Suppose the teacher model contains L layers of Transformer structures, and each layer has H attention head matrices. Taking the l-th layer as an example, for each attention head matrix calculate the statistics of the attention head matrix along the sequence dimension and concatenate the statistics to obtain the global feature vector of the l-th layer, as shown in the following formula:

[0037]

[0038] wherein is the query, key, and value matrix of the h-th head;

[0039]

[0040] wherein are the mean, variance, and entropy of the l-th layer in the h-th head attention matrix, respectively

[0041]

[0042] wherein is the global feature vector of the l-th layer of the teacher model;

[0043] The method maps the output of the m-th layer of the student model FCNN to the teacher model through a projection matrix for feature alignment, as follows:

[0044]

[0045] wherein is the feature matrix projected from the student model to the teacher model, is the hidden output of the m-th layer of the student model, is the projection matrix of the student model;

[0046] In the global interaction module, the distillation loss function is composed of minimizing the cosine distance to calculate the feature alignment loss, as follows:

[0047]

[0048] wherein is the distillation loss function of the global feature extraction module, are the normalization matrices of the feature matrix of the l-th layer of the teacher model and the projected feature matrix of the m-th layer of the student model, respectively;

[0049] S304: Design the complete distillation loss function of the teacher-student model, as follows:

[0050]

[0051] wherein is the total distillation loss function, λ 1 and λ 2 and λ 3 are respectively weight parameters.

[0052] In this solution, training and testing the teacher model and the student model in step S4 specifically includes:

[0053] S401: train the teacher model using the training set, and freeze the parameters of the teacher model after the model parameters converge;

[0054] S402: Use the layer attention distillation method in step S3 to align the convolutional layer outputs of the teacher model and the student model; use the temporal attention distillation method to align the LSTM hidden state distributions of the teacher model and the student model; use the feature alignment distillation method in step S3 to map the Transformer self-attention features of the teacher model to the FCNN fully connected layer of the student model, and perform joint training in combination with the SOH prediction task loss;

[0055] S403: Use the validation set to adjust the hyperparameters of the student model to prevent overfitting;

[0056] S404: Calculate the root mean square error and mean absolute error of the SOH estimation of the student model using the test set to verify the performance of the student model;

[0057] In this solution, during the actual operation described in step S5, online SOH estimation is performed using the SOH estimation model, specifically including:

[0058] S501: deploying the trained student model to the battery management system;

[0059] S502: The lithium battery management system collects the voltage data of the constant current charging stage and the current data of the constant voltage charging stage of the lithium battery, and after preprocessing the data, performs online SOH estimation using the SOH estimation model.

[0060] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0061] The present invention proposes a lithium battery SOH estimation method based on knowledge distillation, which uses a high-precision deep learning model as a teacher model and a lightweight deep learning model as a student model, making full use of the characterization ability of the complex structure of the teacher model and the lightweight advantage of the student model. Under the premise of meeting the high-precision estimation of the lithium battery SOH, the adaptability of the SOH estimation method on embedded devices with low computing power is significantly improved. At the same time, by designing a distillation method between heterogeneous models, the student model can take advantage of the advantages of different architecture models to extract features of battery aging data from three dimensions: local features, temporal degradation memory, and global interactions, thereby extracting multimodal features related to the battery SOH and improving the SOH estimation accuracy. The above two advantages are both conducive to improving the SOH estimation performance of lithium batteries. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and thus should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other relevant drawings can also be obtained based on these drawings.

[0063] Figure 1 A method for estimating the state of health (SOH) of a lithium battery based on knowledge distillation provided in an embodiment of the present invention.

[0064] Figure 2 The voltage data curve during the constant current charging stage of the lithium battery.

[0065] Figure 3 The current data curve during the constant voltage charging stage of the lithium battery. Specific embodiments

[0066] In order to better understand the above objects, features, and advantages of the present invention, the following further details the present invention in conjunction with the drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0067] Many specific details are set forth in the following description to facilitate a thorough understanding of the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0068] For ease of understanding, please refer to Figure 1 , an embodiment of a method for estimating the SOH of a lithium battery based on knowledge distillation provided by the present invention includes:

[0069] S1: Collection of lithium battery data sets, that is, constant current-constant voltage charging and constant current discharging are performed on multiple lithium batteries of the same model. During each charge and discharge process, the voltage data during the constant current charging stage and the current data during the constant voltage charging stage are respectively collected and recorded. After each discharge ends, the SOH value of the lithium battery in this cycle is calculated; preprocessing of the lithium battery aging data set; dividing the above data into a training set, a validation set, and a test set, where the training set is used for joint optimization of the teacher-student model, the validation set is used for hyperparameter tuning, and the test set is used for final performance evaluation;

[0070] S2: Building a teacher-student knowledge distillation model, which is composed of the following three functional modules:

[0071] Local feature extraction module: It includes the convolutional neural network (CNN) of the teacher model and the lightweight CNN of the student model. The depth of the CNN of the teacher model is 1.5 to 2 times that of the student model, and the number of convolutional kernels in each layer is not less than 1.2 times that of the student model;

[0072] Temporal degradation memory module: It includes the Bi-LSTM of the teacher model and the unidirectional LSTM of the student model. The hidden layer dimension of the bidirectional long short-term memory network of the teacher model is 1.5 times that of the student model;

[0073] Global interaction module: It is composed of the Transformer encoder of the teacher model and the FCNN of the student model. The Transformer encoder includes a multi-head self-attention mechanism, and the number of hidden layers of the FCNN does not exceed 3 layers;

[0074] S3: Design the distillation method for each module. The specific steps include: First, use the layer attention distillation method for the local feature extraction module to align the output features of the convolutional layers of the teacher model and the student model; use the temporal attention distillation method for the temporal feature extraction module to align the temporal attention distributions of the Bi-LSTM of the teacher model and the LSTM of the student model through the temporal attention distillation module; use the feature alignment distillation method for the global feature extraction module to map the self-attention features of the Transformer of the teacher model to the fully connected layer of the student model; design the total distillation loss function and iteratively optimize the parameters of the student model;

[0075] S4: Use the training set to independently pre-train the teacher model. After the parameters of the teacher model converge, freeze all the parameters of the teacher model, and perform knowledge transfer training on the student model based on the distillation method in S3; then use the validation set to adjust the hyperparameters of the student model to prevent overfitting; finally, use the test set to verify the performance of the trained student model, and use the root mean square error and the mean absolute value to evaluate the model performance;

[0076] S5: Online estimate the SOH of the lithium battery, that is, during the actual charge and discharge process of the lithium battery, collect the current data in the constant voltage charging stage and the voltage data in the constant current charging stage during the charging process. After preprocessing the current data in the constant voltage charging stage and the voltage data in the constant current charging stage according to the method in S1, input them into the trained student model described in S4 to make it output the SOH estimation value.

[0077] In this solution, the distillation method for designing each module described in step S3 is specifically:

[0078] S301: Design the layer attention distillation method for the local feature extraction module. The layer attention distillation method includes cross-layer attention alignment of the output features of the convolutional layers of the teacher CNN and the student CNN to generate channel-space joint attention weights, and the output of the l-th layer of the teacher CNN Map the student features to the teacher space through a learnable convolutional kernel, as shown in the following formula:

[0079]

[0080] Where is the student feature mapped to the teacher space, and W align is a 1×1 convolutional kernel, is the output of the m-th layer of the student convolutional neural network, and C tea , C stu are the number of channels of the teacher and student convolutional neural networks respectively, and T is the time step;

[0081] Generate the channel weight vector of the layer attention weight by performing global average pooling and global max pooling on the features of the teacher model and the student model, as shown in the following formula:

[0082]

[0083] Where are the global average pooling weight vector and the global max pooling weight vector of the teacher features respectively, are the global average pooling weight vector and the global max pooling weight vector of the student features respectively, GAP and GMP are the global average pooling result and the global max pooling result respectively, and σ is the Sigmoid activation function;

[0084] In the local feature extraction module, the attention map feature is determined by the channel weight vector, as shown in the following formula:

[0085]

[0086] Where M tea , M stu are the attention feature maps after alignment of the teacher model and the student model respectively;

[0087] In the local feature extraction module, the distillation loss function includes a feature map alignment term, as shown in the following formula:

[0088]

[0089] Where, is the distillation loss of the local feature extraction module, M tea [i], M stu[i] are the i-th layer attention feature maps after alignment of the teacher model and the student model; n is the total number of layers of the attention feature maps after alignment of the teacher model and the student model.

[0090] S302: Design a temporal attention distillation method, where the temporal attention distillation method includes encoding the temporal features of the teacher model and the student model, and concatenating the forward hidden state and the backward hidden state of the teacher model into a bidirectional hidden state, as shown in the following formula:

[0091]

[0092] The temporal attention weights of the teacher Bi-LSTM are calculated using only the forward hidden state, and the temporal attention weights of the student LSTM are based on the unidirectional hidden state, as shown in the following formula:

[0093]

[0094] where are the temporal attention weights of the teacher model and the student model respectively are the hidden states of the teacher model Bi-LSTM at time step t and the hidden state of the student model LSTM respectively, v tea and v stu are the learnable context vectors of the teacher model and the student model respectively; T is the total number of time steps of the hidden state of the teacher model Bi-LSTM and the hidden state of the student model LSTM;

[0095] In the temporal degradation memory module, the distillation loss function includes the teacher weighted feature and the student weighted feature, as shown in the following formula:

[0096]

[0097] where is the distillation loss of the temporal feature extraction module, W c is the teacher bidirectional feature compression matrix ||·|| 2 is the L2 norm normalization of the temporal features of the teacher model and the student model;

[0098] S303: Design a global interaction module feature alignment distillation method, where the global interaction module feature alignment distillation method extracts the self-attention features of the teacher model. Assume the teacher model contains L layers of Transformer structures, and each layer has H attention head matrices. Taking the l-th layer as an example, for each attention head matrix calculate the statistics of the attention head matrix along the sequence dimension and concatenate the statistics to obtain the global feature vector of the l-th layer, as shown in the following formula:

[0099]

[0100] Among them is the query, key, and value matrix of the h-th head;

[0101]

[0102] Among them are the mean, variance, and entropy of the l-th layer in the h-th head attention matrix, respectively

[0103]

[0104] Among them is the global feature vector of the l-th layer of the teacher model;

[0105] The method maps the output of the m-th layer of the student model FCNN to the teacher model through a projection matrix for feature alignment, as follows:

[0106]

[0107] Among them is the feature matrix projected from the student model to the teacher model, is the hidden output of the m-th layer of the student model, is the projection matrix of the student model;

[0108] In the global interaction module, the distillation loss function is composed of minimizing the cosine distance to calculate the feature alignment loss, as follows:

[0109]

[0110] Among them is the distillation loss function of the global feature extraction module, are the normalization matrices of the feature matrix of the l-th layer of the teacher model and the projected feature matrix of the m-th layer of the student model, respectively;

[0111] S304: Design the complete distillation loss function of the teacher-student model, as follows:

[0112]

[0113] Among them is the total distillation loss function, λ 1 and λ 2 and λ 3 are respectively weight parameters;

[0114] In this solution, the training and testing of the teacher model and the student model in step S4 specifically include:

[0115] S401: Train the teacher model using the training set, and freeze the parameters of the teacher model after the model parameters converge;

[0116] S402: Use the layer attention distillation method in step S3 to align the convolutional layer outputs of the teacher model and the student model; use the temporal attention distillation method to align the LSTM hidden state distributions of the teacher model and the student model; use the feature alignment distillation method in step S3 to map the teacher model Transformer self-attention features to the FCNN fully connected layer of the student model, and perform joint training in combination with the SOH prediction task loss;

[0117] S403: Adjust the hyperparameters of the student model using the validation set to prevent overfitting;

[0118] S404: Calculate the root mean square error and mean absolute error of the SOH estimation of the student model using the test set to verify the performance of the student model;

[0119] In this solution, during the actual operation described in step S5, use the SOH estimation model for online SOH estimation, specifically including:

[0120] S501: Deploy the trained student model to the battery management system;

[0121] S502: The lithium battery management system collects the voltage data in the constant current charging stage and the current data in the constant voltage charging stage of the lithium battery. After data preprocessing, use the SOH estimation model for online SOH estimation.

[0122] The voltage data curve in the constant current charging stage of the lithium battery is referred to Figure 2 , where the abscissa represents time, and the unit is seconds; the ordinate represents current, and the unit is amperes; as time increases, the charging current gradually decreases until it stabilizes when approaching the full charge state;

[0123] The current data curve in the constant voltage charging stage of the lithium battery is referred to Figure 3 , where the abscissa represents time, and the unit is seconds; the ordinate represents voltage, and the unit is volts; as time increases, the charging voltage gradually increases until it stabilizes when approaching the full charge state.

Claims

1. A lithium battery SOH estimation method based on knowledge distillation, characterized in that: The specific steps are as follows: S1: Lithium battery data set collection, i.e., constant current-constant voltage charging and constant current discharging are performed on multiple lithium batteries of the same model. During each charging and discharging process, the voltage data of the constant current charging stage and the current data of the constant voltage charging stage are collected and recorded respectively in the constant current and constant voltage charging stages. After each discharge, the SOH value of the lithium battery in this cycle is calculated; the lithium battery aging data set is preprocessed; the above data is divided into a training set, a validation set, and a test set, where the training set is used for joint optimization of the teacher-student model, the validation set is used for hyperparameter tuning, and the test set is used for final performance evaluation; S2: Build a teacher-student knowledge distillation model, which consists of the following three functional modules: Local feature extraction module: including the convolutional neural network (CNN) of the teacher model and the lightweight CNN of the student model. The depth of the CNN of the teacher model is 1.5 to 2 times that of the student model, and the number of convolution kernels in each layer is not less than 1.2 times that of the student model. Temporal degradation memory module: including the bidirectional long short-term memory network (Bi-LSTM) of the teacher model and the unidirectional long short-term memory network (LSTM) of the student model. The hidden layer dimension of the bidirectional LSTM network of the teacher model is 1.5 times that of the student model. Global interaction module: The Transformer encoder of the teacher model and the fully connected neural network (FCNN) of the student model are composed of the Transformer encoder including a multi-head self-attention mechanism, and the number of hidden layers of the FCNN does not exceed 3; S3: Design the distillation method for each module. The specific steps include: first, use the layer attention distillation method for the local feature extraction module to align the convolutional layer output features of the teacher model and the student model; use the temporal attention distillation method for the time domain feature extraction module to align the temporal attention distribution of the teacher model Bi-LSTM and the student model LSTM through the temporal attention distillation module; use the feature alignment distillation method for the global feature extraction module to map the self-attention features of the teacher model Transformer to the fully connected layer of the student model; design the total distillation loss function and iteratively optimize the student model parameters; S4: Use the training set to pre-train the teacher model independently. After the parameters of the teacher model converge, freeze all the parameters of the teacher model, and perform knowledge transfer training on the student model based on the distillation method in S3. Then use the validation set to adjust the hyperparameters of the student model to prevent overfitting. Finally, use the test set to verify the performance of the trained student model, and use the root mean square error and mean absolute value to evaluate the model performance. S5: Online estimation of the SOH of the lithium battery, that is, during the actual charging and discharging process of the lithium battery, the current data of the constant voltage charging stage and the voltage data of the constant current charging stage are collected during the charging process, and the current data of the constant voltage charging stage and the voltage data of the constant current charging stage are preprocessed according to the method of S1, and then input into the trained student model described in S4, so that it outputs the SOH estimation value.

2. A lithium battery SOH estimation method based on knowledge distillation according to claim 1, characterized in that: The distillation method between the modules described in step S3 has the following specific steps: S301: Design a layer attention distillation method for the local feature extraction module, wherein the layer attention distillation method includes performing cross-layer attention alignment on the convolutional layer output features of the teacher CNN and the student CNN, generating channel-space joint attention weights, and outputting the teacher CNN layer l. The student features are mapped to the teacher space through a learnable convolution kernel, as follows: in is the student feature mapped to the teacher space, W align is a 1×1 convolution kernel, is the output of the mth layer of the student convolutional neural network, C tea , C stu are the number of channels of the teacher and student convolutional neural networks, respectively, and T is the time step; By performing global average pooling and global maximum pooling on the features of the teacher model and the student model, the channel weight vector of the layer attention weight is generated, as shown in the following formula: in are the global average pooling weight vector and the global maximum pooling weight vector of the teacher features, respectively. are the global average pooling weight vector and global maximum pooling weight vector of student features, GAP and GMP are the global average pooling results and global maximum pooling results, respectively, and σ is the Sigmoid activation function; In the local feature extraction module, the attention map feature is determined by the channel weight vector, as shown in the following formula: Among them, M tea 、M stu They are the attention feature maps after the teacher model and the student model are aligned; In the local feature extraction module, the distillation loss function includes the feature map alignment term, as follows: in, is the distillation loss of the local feature extraction module, M tea [i] M stu [i] is the i-th layer of attention feature map after the teacher model and the student model are aligned; n is the total number of layers of attention feature map after the teacher model and the student model are aligned; S302: Design a temporal attention distillation method, which includes encoding the temporal features of the teacher model and the student model, and converting the forward hidden state of the teacher model into and the backward hidden state The splicing is a bidirectional hidden state, as follows: The temporal attention weight of the teacher Bi-LSTM only uses the forward hidden state to calculate the weight, and the temporal attention weight of the student LSTM is based on the unidirectional hidden state, as shown in the following formula: in are the temporal attention weights of the teacher model and the student model, are the hidden state of the teacher model Bi-LSTM at time step t and the hidden state of the student model LSTM, respectively. tea 、v stu are the learnable context vectors of the teacher model and the student model respectively; T is the total time step of the hidden state of the teacher model Bi-LSTM and the hidden state of the student model LSTM; In the temporal degradation memory module, the distillation loss function includes the teacher weighted features and the student weighted features, as shown in the following formula: in is the distillation loss of the temporal feature extraction module, W c is the teacher's bidirectional feature compression matrix ||·||2 is the L2 norm normalization of the temporal features of the teacher model and the student model; S303: Design a global interaction module feature alignment distillation method. The global interaction module feature alignment distillation method extracts the self-attention features of the teacher model. Assume that the teacher model includes L layers of Transformer structure, each layer has H attention head matrices, and take the lth layer as an example. Take each attention head matrix Calculate the statistics of the attention head matrix along the sequence dimension and concatenate the statistics to obtain the global feature vector of the lth layer, as shown in the following formula: in is the query, key, and value matrix of the hth head; in are the mean, variance, and entropy of the lth layer in the hth head attention matrix. in is the global feature vector of the teacher model at layer l; The method outputs the mth layer of the student model FCNN and maps it to the teacher model through a projection matrix for feature alignment, as shown in the following formula: in is the feature matrix projected from the student model to the teacher model, is the hidden output of the mth layer of the student model, is the projection matrix of the student model; In the global interaction module, the distillation loss function is composed of minimizing the cosine distance and calculating the feature alignment loss, which is as follows: in is the distillation loss function of the global feature extraction module, They are the normalized matrices of the l-th layer feature matrix of the teacher model and the m-th layer projection feature matrix of the student model respectively; S304: Design the complete distillation loss function of the teacher-student model, as follows: in is the total distillation loss function, λ1, λ2, and λ3 are The weight parameter of .

3. A lithium battery SOH estimation method based on knowledge distillation according to claim 1, characterized in that: The specific steps of training the teacher model and the student model in step S4 are as follows: S401: train the teacher model using the training set, and freeze the parameters of the teacher model after the model parameters converge; S402: Use the layer attention distillation method in step S3 to align the convolutional layer outputs of the teacher model and the student model; use the temporal attention distillation method to align the LSTM hidden state distributions of the teacher model and the student model; use the feature alignment distillation method in step S3 to map the Transformer self-attention features of the teacher model to the FCNN fully connected layer of the student model, and perform joint training in combination with the SOH prediction task loss; S403: Use the validation set to adjust the hyperparameters of the student model to prevent overfitting; S404: Use the test set to calculate the root mean square error and mean absolute error of the student model SOH estimation to verify the performance of the student model.

4. A lithium battery SOH estimation method based on knowledge distillation according to claim 1, characterized in that: During the actual operation described in step S5, the SOH lightweight estimation model is used to perform online SOH estimation, and the specific steps are as follows: S501: deploying the trained student model to the battery management system; S502: The lithium battery management system collects the voltage data of the constant current charging stage and the current data of the constant voltage charging stage of the lithium battery, and after preprocessing the data, performs online SOH estimation using the SOH estimation model.

Images