Battery health state prediction method, model training method and device

Abstract

The invention provides a battery health state prediction method and device and a model training method and device, and relates to the technical field of batteries. According to the method, state feature extraction is realized through the feature extraction model with strong feature extraction capability, potential features in the operation state data of the battery can be better extracted, the health state is predicted through the self-attention model, important information in the state features can be better concerned, and the prediction accuracy of the health state is improved. And the global information is better captured, so that more and more important information can be effectively utilized to realize prediction, and the prediction precision is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of battery technology, and in particular to a battery health status prediction method, model training method and device. Background Art

[0002] In the field of battery technology, accurate assessment of the battery's state of health (SOH) is crucial to the safety and power of electric vehicles. However, due to the complex and irreversible physical or chemical side reactions that occur during battery aging, high-precision SOH prediction has become a challenging task.

[0003] At present, the common SOH prediction methods mainly include experimental methods or physical model-based prediction methods. The experimental methods have strict requirements on the test environment and equipment and are difficult to apply online. The physical model-based methods are difficult to establish a high-fidelity physical model to describe the aging process of batteries under various complex operating conditions. Therefore, these methods are difficult to guarantee the prediction accuracy. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a battery health status prediction method, model training method and device to improve prediction accuracy.

[0005] In a first aspect, an embodiment of the present application provides a method for predicting a battery health state, the method comprising:

[0006] The feature extraction model in the prediction model is used to extract the features of the battery operation status data to obtain the status features;

[0007] The health state prediction of the battery is performed according to the state characteristics through the self-attention model in the prediction model to obtain the health state prediction result of the battery.

[0008] In the above implementation process, state feature extraction is realized through a feature extraction model with powerful feature extraction capabilities, which can better extract potential features in the battery's operating status data. The self-attention model is used to predict the health status, which can better focus on the important information in the state features and better capture the global information therein, so that more and more important information can be effectively used to realize prediction, thereby effectively improving the prediction accuracy.

[0009] Optionally, the operating status data of the battery includes data of multiple charge and discharge cycles;

[0010] The performing health state prediction according to the state feature by using the self-attention model in the prediction model to obtain the health state prediction result of the battery includes:

[0011] The self-attention model in the prediction model is used to predict the health status according to the state characteristics corresponding to the data of each charge and discharge cycle, so as to obtain the health status prediction result of each charge and discharge cycle.

[0012] In the above implementation process, batch prediction of the health status of multiple charging and discharging cycles can be achieved to improve the prediction efficiency, and the correlation between the data of each charging and discharging cycle can be utilized to improve the prediction accuracy.

[0013] Optionally, the operating status data of the battery includes data of multiple charge and discharge cycles; and the performing health status prediction according to the status characteristics by the self-attention model in the prediction model to obtain the health status prediction result of the battery includes:

[0014] The self-attention model in the prediction model reduces the value of at least one target element in the intermediate features obtained to obtain the target feature, wherein the intermediate feature refers to the state feature extracted by the self-attention model, and each target element represents the correlation between the data of the current charge and discharge cycle corresponding to it and the data of a subsequent charge and discharge cycle;

[0015] The health state prediction of the battery is performed based on the target feature through the self-attention model to obtain a health state prediction result of the battery.

[0016] In the above implementation process, the self-attention model is used to reduce the dependence of the data of the current charge and discharge cycle on the data of subsequent charge and discharge cycles, thereby effectively reducing the influence of the subsequent charge and discharge cycle data on the prediction results of the current charge and discharge cycle and improving the prediction accuracy.

[0017] Optionally, reducing the value of at least one target element in the intermediate features obtained by the self-attention model in the prediction model to obtain the target feature includes:

[0018] The value of at least one target element in the intermediate feature is set to 0 through the self-attention model to obtain the target feature.

[0019] In the above implementation process, the value of the target element is set to 0, so that the prediction of the current charge and discharge cycle depends only on the data of the previous charge and discharge cycle, which can effectively reduce the impact of subsequent charge and discharge cycle data on the prediction result of the current charge and discharge cycle and improve the prediction accuracy.

[0020] Optionally, the step of setting the value of at least one target element in the intermediate feature to 0 through the self-attention model to obtain the target feature includes:

[0021] A mask is applied to the intermediate feature through the self-attention model to obtain a target feature, wherein the element value at a position in the mask corresponding to at least one target element in the intermediate feature is 0.

[0022] In the above implementation process, by applying a mask to the intermediate features, the dependency of the current charge and discharge cycle data on the subsequent charge and discharge cycle data during prediction can be quickly eliminated, which reduces the computational complexity and improves the efficiency.

[0023] Optionally, the intermediate feature refers to a query matrix Q, a key matrix K or a value matrix V obtained by the self-attention model in a self-attention operation process;

[0024] Alternatively, the intermediate feature refers to a feature obtained based on at least two matrix operations among the query matrix Q, the key matrix K, and the value matrix V.

[0025] In the above implementation process, some intermediate features are flexibly selected for processing, so as to achieve the effect of reducing the influence of subsequent charge and discharge cycle data on the current charge and discharge cycle prediction result.

[0026] Optionally, the feature extraction model is a convolutional neural network model. The convolutional neural network model has a powerful feature extraction capability, and thus can extract more and more detailed state features from the running state data.

[0027] In a second aspect, an embodiment of the present application provides a model training method, the method comprising:

[0028] Get the self-attention model that has been initially trained;

[0029] The network weights of the initially trained self-attention model are fine-tuned through a low-rank adaptive mechanism to obtain a trained self-attention model.

[0030] In the above implementation process, the network weights of the initially trained self-attention model are fine-tuned through a low-rank adaptive mechanism, which can reduce the impact of the powerful fitting ability of the self-attention model on the training accuracy, so that the trained self-attention model obtained after fine-tuning can more accurately predict the health status.

[0031] Optionally, fine-tuning the network weights of the preliminarily trained self-attention model by a low-rank adaptive mechanism to obtain a trained self-attention model includes:

[0032] Obtain a weight adjustment matrix corresponding to a weight matrix of a model target layer, wherein the model target layer is a layer whose parameters need to be fine-tuned in the self-attention model that has been preliminarily trained;

[0033] The element values ​​in the weight adjustment matrix are updated to adjust the network weights of the target layer of the model, and a trained self-attention model is obtained after the adjustment is completed.

[0034] In the above implementation process, the network weights of the model are updated by updating the element values ​​in the weight adjustment matrix, which can effectively reduce the amount of training of network parameters and improve training efficiency.

[0035] Optionally, the step of obtaining a weight adjustment matrix corresponding to a weight matrix of a target layer of the model includes:

[0036] Initialize the weight adjustment matrix corresponding to the weight matrix of the target layer of the model to obtain the initialized weight adjustment matrix.

[0037] In the above implementation process, initializing the weight adjustment matrix can reduce the damage to the performance of the self-attention model that has completed the initial training, that is, it can reduce the problem of the model falling into the local optimal solution at the beginning of training.

[0038] Optionally, the weight adjustment matrix includes two low-rank matrices; and initializing the weight adjustment matrix corresponding to the weight matrix of the target layer of the model to obtain the initialized weight adjustment matrix includes:

[0039] One of the two low-rank matrices in the weight adjustment matrix corresponding to the weight matrix of the target layer of the model is initialized to 0, and the other low-rank matrix is ​​randomly initialized to obtain the initialized weight adjustment matrix.

[0040] In the above implementation process, if both low-rank matrices are randomly initialized, the output of the model may change greatly during the initial training, which will damage the original performance of the model. Therefore, initializing one of the low-rank matrices to 0 can effectively reduce the damage to the original performance of the model.

[0041] Optionally, updating the element values ​​in the weight adjustment matrix includes:

[0042] After each update of the element values ​​in the weight adjustment matrix, the variance of the part of the output of the model target layer corresponding to the weight adjustment matrix is ​​adjusted to a set value, and the set value is determined based on the variance of the probability distribution obeyed by the input data.

[0043] In the above implementation process, each time after updating the element value in the weight adjustment matrix, the variance of the part of the output of the model target layer corresponding to the weight adjustment matrix is ​​controlled within the set value, which can reduce the problem of model sub-optimization caused by the large difference in training accuracy each time.

[0044] Optionally, the weight adjustment matrix includes two low-rank matrices; and adjusting the variance of a portion of the output of the target layer of the model corresponding to the weight adjustment matrix to a set value includes:

[0045] The two low-rank matrices are scaled respectively so that the variance of the part of the output of the model target layer corresponding to the weight adjustment matrix is ​​adjusted to a set value.

[0046] Optionally, the model target layer includes a network layer of multiple self-attention units in the self-attention model that has been initially trained.

[0047] In the above implementation process, the network weights of the network layers in the self-attention unit are fine-tuned, because these network layers are the key layers in the self-attention model, and the self-attention mechanism is the key to influencing the prediction effect. Therefore, fine-tuning these layers in the self-attention unit not only reduces the amount of calculation, but also reduces the problem of model overfitting, which can effectively improve the training accuracy of the model.

[0048] Optionally, the weight adjustment matrices corresponding to the network layers in the multiple self-attention units include: a weight adjustment matrix corresponding to at least one of the weight matrix of the network layer used to combine the calculation results of multiple self-attention heads, the weight matrix of the network layer used to calculate the query matrix, the weight matrix of the network layer used to calculate the key matrix, and the weight matrix of the network layer used to calculate the value matrix.

[0049] In the above implementation process, you can choose to fine-tune the network weights of some network layers in the self-attention unit, which can reduce the update parameters in the training process and improve the training efficiency. You can choose to fine-tune the network weights of all network layers in the self-attention unit to improve the training effect.

[0050] In a third aspect, an embodiment of the present application provides a battery health status prediction device, the device comprising:

[0051] A feature extraction module is used to extract features from the battery operation status data through a feature extraction model in the prediction model to obtain status features;

[0052] A prediction module is used to predict the health state of the battery according to the state characteristics through a self-attention model in the prediction model to obtain a health state prediction result of the battery.

[0053] In a fourth aspect, an embodiment of the present application provides a model training device, the device comprising:

[0054] Model acquisition module, used to obtain the self-attention model that has been initially trained;

[0055] The weight adjustment module is used to fine-tune the network weights of the self-attention model that has been preliminarily trained through a low-rank adaptive mechanism to obtain a trained self-attention model.

[0056] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect or the second aspect are executed.

[0057] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method provided in the first aspect or the second aspect are performed.

[0058] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 A flowchart of a method for predicting the health status of a battery provided in an embodiment of the present application;

[0061] Figure 2 A schematic diagram of the overall structure of a prediction model provided in an embodiment of the present application;

[0062] Figure 3 A schematic diagram of the structure of a convolutional neural network model provided in an embodiment of the present application;

[0063] Figure 4 A schematic diagram of the structure of a self-attention model provided in an embodiment of the present application;

[0064] Figure 5 A flowchart of a model training method provided in an embodiment of the present application;

[0065] Figure 6 A schematic diagram of a structure for fine-tuning using a low-rank adaptive mechanism provided in an embodiment of the present application;

[0066] Figure 7 A structural block diagram of a battery health status prediction device provided in an embodiment of the present application;

[0067] Figure 8 A structural block diagram of a model training device provided in an embodiment of the present application;

[0068] Fig. 9 A schematic diagram of the structure of an electronic device for executing a battery health status prediction method or a model training method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present application.

[0070] It should be noted that the terms "system" and "network" in the embodiments of the present invention can be used interchangeably. "Multiple" means two or more. In view of this, in the embodiments of the present invention, "multiple" can also be understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / ", unless otherwise specified, generally indicates that the previously associated objects are in an "or" relationship.

[0071] In order to predict the health status of the battery, existing solutions are generally based on experiments or physical models. However, these methods rely on the test environment and it is difficult to ensure the prediction accuracy.

[0072] Therefore, an embodiment of the present application provides a method for predicting the health status of a battery. In this solution, a feature extraction model with powerful feature extraction capabilities is used to realize state feature extraction, which can better extract potential features in the battery's operating status data, and a self-attention model is used to realize the prediction of the health status. It can better pay attention to the important information in the state characteristics, and better capture the global information therein, so that more and more important information can be effectively utilized to realize prediction, thereby effectively improving the prediction accuracy.

[0073] Please refer to Figure 1 , Figure 1 A flowchart of a method for predicting the health status of a battery provided in an embodiment of the present application, the method comprising the following steps:

[0074] Step S110: extracting features from the battery operating status data using a feature extraction model in the prediction model to obtain status features.

[0075] The prediction model in this scheme includes a feature extraction model and a self-attention model. The output of the feature extraction model is connected to the input of the self-attention model. The overall structure diagram of the prediction model is as follows: Figure 2 shown.

[0076] In some embodiments, the feature extraction model may be a convolutional neural network model. Since the convolutional neural network model has a powerful feature extraction capability, it can better extract potential features from the battery operating status data.

[0077] In order to realize feature extraction, the convolutional neural network model in this scheme may include multiple convolutional layers and output layers, and its structure diagram can be shown as follows: Figure 3 shown.

[0078] In this solution, the battery operating status data is used as the input of the convolutional neural network model. In some embodiments, the battery operating status data includes at least one of the current I, voltage V, and temperature T. Of course, in practical applications, other data may also be included, such as the battery state of charge.

[0079] In order to obtain higher prediction accuracy, more operating status data can be obtained and input into the prediction model to realize the prediction of health status, because these operating status data can reflect the health status of the battery from many aspects.

[0080] If the input data of the convolutional neural network model includes VIT curve data formed by current, voltage, and temperature, these data can be represented by X. The purpose of the convolutional neural network model is to learn a mapping function f θ (·) is used to map the input X into an intermediate feature, namely, a state feature, represented by Z = f θ (X).

[0081] Among them, the parameters of each convolution layer may include convolution kernel, stride and padding, i.e. conv(kernel_size,stride,padding), Figure 3 If the hidden size is not specified, it can default to 256. Figure 3 The convolutional neural network model in can have 7 convolutional layers, of which 6 convolutional layers have a stride of 2 and 1 convolutional layer has a stride of 1. After the input data is convolved through 7 convolutional layers in sequence, it is compressed into a vector with a spatial size of 1 and a feature dimension of 256, and finally the state features are output through the output layer (Z).

[0082] It is understandable that in other implementations, the specific structure of the convolutional neural network model can be flexibly set, and the number and parameters of its convolutional layers can also be flexibly set according to actual conditions. The convolutional neural network model has a strong feature extraction capability, and it is insensitive to time and location information, so it can also extract features well for historical data, making the extracted features more comprehensive and more useful. In addition, the input of the convolutional neural network model has no dependence on the output at the previous time point, and parallel computing can be achieved, making feature extraction more efficient.

[0083] In other embodiments, the feature extraction model in the present application may also be other models that can perform feature extraction, such as a recurrent neural network model, a long short-term memory network model, etc., which are not listed here one by one.

[0084] Step S120: The health status is predicted according to the state characteristics by using the self-attention model in the prediction model to obtain the health status prediction result of the battery.

[0085] The self-attention model is a neural network model that uses the self-attention mechanism to implement specific functions. For example, the self-attention model can be a Transformer model and its variants. When the self-attention model is used to predict health status, since the self-attention model can better focus on the global information of the state features, better understand the contextual information in the sequence data, and capture the complex features by calculating the correlation between the features, it can better extract more information from the state features to achieve prediction, thereby achieving higher prediction accuracy.

[0086] It can be understood that the output of the self-attention model can be data representing the health status of the battery, such as using a value between 0 and 1 to represent the health of the battery, where a higher value indicates a healthier state and a lower value indicates an unhealthier state. In this way, the health status of the battery can be determined directly based on the output of the self-attention model. Alternatively, the output of the self-attention model can also be a value of 0 or 1 directly, where a value of 0 indicates that the battery is in a poor health state and a value of 1 indicates that the battery is in a good health state. In this way, the operation and maintenance personnel can directly perform corresponding maintenance on batteries in poor health states based on the output value.

[0087] In the above implementation process, state feature extraction is realized through a feature extraction model with powerful feature extraction capabilities, which can better extract potential features in the battery's operating status data. The self-attention model is used to predict the health status, which can better focus on the important information in the state features and better capture the global information therein, so that more and more important information can be effectively used to realize prediction, thereby effectively improving the prediction accuracy.

[0088] On the basis of any embodiment of the present application, in order to predict the health status of the battery in multiple charge and discharge cycles, the battery's operating status data may include data from multiple charge and discharge cycles, and the data from each charge and discharge cycle may include at least one of current, voltage, and temperature. Then, the feature extraction model may be used to perform feature extraction on the data from multiple charge and discharge cycles of the battery, respectively, to obtain state features corresponding to the data from each charge and discharge cycle, and then a self-attention model may be used to perform health status prediction based on the state features corresponding to the data from each charge and discharge cycle, to obtain a health status prediction result for each charge and discharge cycle.

[0089] For example, the data of each charge and discharge cycle can be used as a set of input data to be input into a convolutional neural network model, such as Figure 2 The cycle_n shown in the figure represents the data of the nth charge and discharge cycle. The convolutional neural network model can extract the state features of the data of each charge and discharge cycle in batches and in parallel, and then input the state features of the data of each charge and discharge cycle into the self-attention model. The self-attention model can also predict the health status of the data of each charge and discharge cycle in batches, thereby realizing the prediction of the health status under multiple charge and discharge cycles, thereby improving the prediction efficiency.

[0090] It can be understood that if in some embodiments, the operating status data of the battery includes data of a charge and discharge cycle, then the data input into the convolutional neural network model can be data collected at various time points during the charge and discharge cycle, such as voltage, current, temperature and other data collected at various time points. In this way, this time series data can be input into the convolutional neural network model, and the state features corresponding to each time point can be extracted through the convolutional neural network model. Then, the health status of the battery under the charge and discharge cycle can be predicted based on these state features through the self-attention model.

[0091] In some embodiments, the self-attention model can also predict an overall result based on the state characteristics of multiple charge and discharge cycle data, that is, the overall health status of the battery can be predicted by combining the data of multiple charge and discharge cycles, so that the health status of the battery during this period can be predicted.

[0092] Alternatively, after the self-attention model outputs prediction results for data of multiple charge and discharge cycles, the final prediction result can be obtained based on the multiple prediction results.

[0093] For example, multiple prediction results can be summed up and then normalized, and the obtained processing result can be used as the final prediction result. The above prediction results are represented by a value between 0 and 1, so the value obtained by summing up multiple prediction results may be greater than 1, so normalization is performed to obtain the final prediction result.

[0094] In the above implementation process, batch prediction of the health status of multiple charging and discharging cycles can be achieved to improve the prediction efficiency, and the correlation between the data of each charging and discharging cycle can be utilized to improve the prediction accuracy.

[0095] Based on the above embodiments, the self-attention model of the present application can be implemented using a Transformer model, such as an encoder and a decoder, the encoder includes multiple encoding blocks, and the decoder also includes multiple encoding blocks. When predicting the health status of the battery, the state features can be input into multiple encoding blocks to obtain a coding information matrix, and then the coding information matrix is ​​passed to the decoder. Multiple decoding blocks will predict the health status based on the coding information matrix, and finally output the prediction results.

[0096] In order to improve the prediction efficiency, the structure of the above-mentioned Transformer model can be simplified in this application. For example, the Transformer model in this solution can only include the encoder part. Specifically, the self-attention model in this application can include a position embedding layer and multiple self-attention units, each of which includes a multi-head self-attention module and a forward propagation module. The specific structure of the self-attention model in this application can be as follows: Figure 4 shown.

[0097] In the scenario where the feature extraction model is a convolutional neural network model, since the convolutional neural network model does not consider the relationship between the data of each charge and discharge cycle when extracting features, the relationship between the data of each charge and discharge cycle can be extracted through the self-attention model. The position embedding layer can use position encoding to represent the position where each state feature appears in the charge and discharge cycle. Because the self-attention model in this scheme does not use the structure of the recurrent neural network model (RNN), but uses global information, it cannot use the sequential information of the data of each charge and discharge cycle, which is very important for the prediction result. Therefore, position encoding is used in the Transformer model to retain the relative or absolute position of the data of each charge and discharge cycle in the data sequence.

[0098] Therefore, when predicting the health status, the position embedding layer can be used to encode the status features to obtain the encoded features. In each self-attention unit, the multi-head self-attention module is used to calculate the attention map based on the output features of the previous self-attention unit, and the self-attention operation is performed according to the attention map to obtain the features after the operation. The forward propagation module is used to perform forward propagation based on the features after the operation to obtain the output features of the current self-attention unit, where the input features of the first self-attention unit are the above-mentioned encoded features, and the output features of the last self-attention unit are the output features of the self-attention model. Then, the health status prediction result of the battery can be predicted based on the output features of the last self-attention unit.

[0099] The position embedding layer here is Figure 4 In the Positional Encoding, in practical applications, the position feature can be obtained through the corresponding calculation formula, such as:

[0100] PE (pos,2i) =sin(pos / 10000 2i / d );

[0101] PE (pos,2i) =cos(pos / 10000 2i / d );

[0102] Among them, PE represents the position feature, pos represents the position of the current charge and discharge cycle data in multiple charge and discharge cycle data, d represents the dimension of PE, 2i represents the even dimension, and 2i+1 represents the odd dimension.

[0103] The input state features are positionally encoded to obtain the corresponding representation vector, that is, the encoded features. The representation vector can be input into the first self-attention unit. The calculation formula of the representation vector can be expressed as follows:

[0104]

[0105] For the feature Z′ output by the lth self-attention unit l :

[0106] Z′ l =LN(MSA(Z l-1 )+Z l-1 )

[0107] For the output feature Z of the l-th self-attention unit l :

[0108] Z l =LN(FF(Z′ l )+Z′ l )

[0109] Among them, l = 1,...,L, the output feature Z of the last self-attention unit L That is, as the output feature of the self-attention model, Figure 4 There are 6 self-attention units shown in FIG. 1 . It can be understood that the number of self-attention units in practical applications can be flexibly set. The output features of the self-attention model can be output as health status prediction results after passing through a Linear layer, as shown below:

[0110] [SOH1,SOH2,…SOH n ]=Linear(LN(Z L ))

[0111] It should be noted that the LN layer and the Linear layer in the above formula are Figure 4 It is not shown in the figure, in some embodiments, the LN layer can also be omitted.

[0112] In the above implementation process, the self-attention unit can better extract important information from the state features and capture the correlation between the data of each charging and discharging cycle, so that this information can be effectively used to predict the health status and improve the prediction accuracy.

[0113] Based on the above embodiment, the multi-head self-attention module includes a Multi-Head Attention layer (corresponding to the MSA in the above self-attention unit expression) and an Add&Norm layer. The Multi-Head Attention layer is composed of multiple Self-Attentions.

[0114] The Add&Norm layer consists of two parts, Add and Norm, and its calculation formula is as follows:

[0115] LN(X+MSA(X));

[0116] LN(X+FF(X));

[0117] Among them, X represents the input of Multi-Head Attention or Feed Forward, and MultiHeadAttention(X) and FeedForward(X) represent the output.

[0118] Add represents residual connection (corresponding to the addition in the self-attention unit expression above), that is, X+MultiHeadAttention(X), which is usually used to solve the problem of multi-layer network training, prevent network degradation, and allow the network to only focus on the current difference. Norm represents Layer Normalization (corresponding to LN in the self-attention unit expression above). The Norm layer converts the input of each layer of neurons into the same mean and variance, which can speed up convergence.

[0119] The forward propagation module includes the Feed Forward layer (corresponding to FF in the self-attention unit expression above) and the Add&Norm layer. For example, the Feed Forward layer can be a two-layer fully connected layer. The activation function of the first layer is Relu, and the second layer does not use an activation function. The corresponding formula is as follows:

[0120] max(0,XW1+b1)W2+b2;

[0121] Wherein, X represents the input, that is, the feature after operation in the above embodiment, W1 and W2 represent the network weights of the layer, b1 and b2 represent biases, and the dimension of the output feature finally obtained by the Feed Forward layer is consistent with X.

[0122] The focus of the multi-head self-attention module is Self-Attention, which requires the query matrix Q, key matrix K and value matrix V during calculation. In fact, Q, K, and V are obtained by linearly changing the input of Self-Attention.

[0123] If the input of Self-Attention is represented by matrix X, the linear change matrices WQ, WK, and WV can be used to calculate Q, K, and V. After obtaining the matrices Q, K, and V, the output of Self-Attention can be calculated. The calculation formula is as follows:

[0124]

[0125] Among them, d k Indicates the number of columns of matrices Q and K, that is, the vector dimension. The formula calculates the inner product of each row vector of matrices Q and K. In order to prevent the inner product from being too large, it is divided by d k The square root of .

[0126] On the basis of the above embodiment, when predicting the health status, it is hoped that the prediction of the current cycle depends only on the previous cycle, but the self-attention model does not automatically learn this temporal relationship. Therefore, in order to reduce the dependence of the current cycle on the future cycle as much as possible, the specific implementation method is: the value of at least one target element in the obtained intermediate feature is reduced by the self-attention model to obtain the target feature, wherein the intermediate feature refers to the state feature obtained by the self-attention model, and each target element represents the correlation between the data of the corresponding current charge and discharge cycle and the data of a subsequent charge and discharge cycle. Then, the health status can be predicted based on the target feature through the self-attention model to obtain the health status prediction result of the battery.

[0127] It can be understood that the way to reduce the value of the target element in the intermediate feature here can be to multiply the value of the target element by a coefficient less than 1, or to subtract a specific value from the value of the target element. The specific methods are not listed here one by one, as long as the correlation between the data of the current charge and discharge cycle and the data of a subsequent charge and discharge cycle is reduced.

[0128] In the above implementation process, the self-attention model is used to reduce the dependence of the data of the current charge and discharge cycle on the data of subsequent charge and discharge cycles, thereby effectively reducing the influence of the subsequent charge and discharge cycle data on the prediction results of the current charge and discharge cycle and improving the prediction accuracy.

[0129] On the basis of the above embodiment, in order to maximize the elimination of the dependence of the prediction of the current charging and discharging cycle on the subsequent charging cycle data and achieve better prediction results, the value of at least one target element in the intermediate feature can be set to 0 through the self-attention model to obtain the target feature.

[0130] Here, directly setting the value of the target element to 0 means changing the value of the target element to 0, which can make the correlation between the data of the current charge and discharge cycle and the data of a subsequent charge and discharge cycle become 0, so that the prediction of the current charge and discharge cycle only depends on the data of the previous charge and discharge cycle, which can effectively reduce the influence of the subsequent charge and discharge cycle data on the prediction result of the current charge and discharge cycle and improve the prediction accuracy.

[0131] On the basis of the above embodiment, in order to quickly set the value of the target element to 0, a mask can be applied to the intermediate feature through the self-attention model to obtain the target feature, where the element value at the position corresponding to at least one target element in the intermediate feature in the mask is 0.

[0132] It can be understood that multiplying the intermediate feature by a mask can be understood as multiplying the intermediate feature by the mask, where the mask refers to elements containing element values ​​0 and 1, and the element value at the position corresponding to at least one target element in the intermediate feature is 0. In this way, after multiplying the intermediate feature by the mask, the element value at the position corresponding to the target element in the target feature becomes 0, thereby eliminating the correlation between the data of the current charging and discharging cycle and the data of the subsequent charging and discharging cycle.

[0133] In the above implementation process, by applying a mask to the intermediate features, the dependency of the current charge and discharge cycle data on the subsequent charge and discharge cycle data during prediction can be quickly eliminated, which reduces the computational complexity and improves the efficiency.

[0134] Based on the above embodiments, the intermediate features may refer to the query matrix Q, the key matrix K or the value matrix V obtained by the self-attention model during the self-attention operation process.

[0135] Alternatively, the intermediate features refer to features obtained based on at least two matrix operations among the query matrix Q, the key matrix K, and the value matrix V.

[0136] For example, the output features of the Self-Attention obtained by the self-attention model during the calculation process can be expressed as:

[0137]

[0138] Therefore, according to this formula, the intermediate feature can refer to any variable parameter in the formula, i.e., Q, K, V, or a feature calculated based on at least two parameters of Q, K, and V, such as QK, QK T , or wait.

[0139] Among them, for the convenience of calculation, the intermediate feature can be the attention map obtained by calculation, and the attention map can be expressed as:

[0140]

[0141] The correlation between cycles (charge and discharge cycle data) is characterized by inner product operations, considering the correlation of the i-th cycle to other cycles (including the i-th cycle itself):

[0142]

[0143] In the above formula, a i That is, it represents the element corresponding to the i-th cycle in matrix A.

[0144] In order to explicitly erase the correlation between the current cycle (i.e., the i-th cycle) and the subsequent cycles, for the values ​​after the i+1th cycle, the element representing the correlation between the i-th cycle and the i+1th cycle can be explicitly set to 0. For example, when traversing the elements in A, if the column number of the element is greater than the row number of the element, it is set to 0. In other words, the target element in the initial attention map can be directly set to 0 to eliminate the correlation between the current cycle and the subsequent cycles.

[0145] Alternatively, in other implementations, a mask M may be applied to the attention map, where the mask is used to indicate that the target elements in the attention map are set to 0:

[0146]

[0147] Among them, m i is the i-th row in mask M, indicating the i-th row a of mask A i The corresponding row mask, is 1, is 0.

[0148] The target feature obtained after applying the mask can be expressed as:

[0149]

[0150] Here, the mask M is a lower triangular matrix:

[0151]

[0152] In the above implementation process, some intermediate features are flexibly selected for processing, so as to achieve the effect of reducing the influence of subsequent charge and discharge cycle data on the current charge and discharge cycle prediction result.

[0153] Based on the above embodiment, the prediction model obtained above can be pre-trained. The prediction model uses a large amount of sample data during training. However, since the operating conditions of various batteries are different, this will cause the prediction model trained under one operating condition to perform poor health prediction for batteries under another operating condition, and the time and resources consumed in retraining the model are relatively large. The present application scheme considers fine-tuning the network weights of the prediction model that has been preliminarily trained to reduce training resources.

[0154] However, due to the powerful fitting ability of the Transformer model, direct fine-tuning will lead to serious overfitting. Therefore, this scheme uses a low-rank adaptive mechanism to fine-tune the network weights of the self-attention model.

[0155] Please refer to Figure 5 , Figure 5 A flowchart of a model training method provided in an embodiment of the present application, the method comprising the following steps:

[0156] Step S210: Obtain the self-attention model that has been initially trained.

[0157] The preliminary training here can be obtained by using the battery data under other working conditions as described in the above embodiment, or it can also be obtained by using the battery data under any working condition. The process of the preliminary training can be performed by the local device (referring to the execution Figure 5 If the model training method is performed by other devices other than the device for performing the model training method, then when fine-tuning the preliminarily trained model, the preliminarily trained model can be obtained from the other devices, and the preliminarily trained model of the other devices can be stored in the local device in advance, and then when fine-tuning, the preliminarily trained model can be directly read from the memory of the local device. Of course, the preliminarily training process can also be performed by the electronic device that performs the health status prediction method.

[0158] It should be noted that the model for preliminary training can be the entire prediction model or a single self-attention model. For example, during the preliminary training, the prediction model can be trained as a whole, or the feature extraction model and the self-attention model in the prediction model can be trained separately. If the prediction model is trained as a whole, then the self-attention model obtained after the preliminary training refers to the self-attention model in the prediction model after the preliminary training, that is, it is necessary to obtain the prediction model after the preliminary training first, and then fine-tune the network weights of the self-attention model in the prediction model separately.

[0159] Step S220: fine-tune the network weights of the initially trained self-attention model through a low-rank adaptive mechanism to obtain a trained self-attention model.

[0160] It can be understood that in the present application, the network weights of the feature extraction model in the prediction model can also be fine-tuned. After research, it is found that fine-tuning the network weights of the feature extraction model has little effect on the subsequent prediction effect. Therefore, in order to improve the training efficiency, only the network weights of the self-attention model need to be fine-tuned. Of course, if it is necessary to fine-tune the network weights of the feature extraction model, the fine-tuning method is similar to the method of fine-tuning the network weights of the self-attention model, which will not be repeated here. In the embodiment of the present application, the fine-tuning of the network weights of the self-attention model is used as an example to illustrate.

[0161] Among them, the low-rank adaptive mechanism, namely LoRA, is based on the idea of ​​decomposing the original weight matrix into the product of two low-rank matrices, which can greatly reduce the number of parameters. In other words, LoRA freezes the parameters of the initially trained self-attention model by injecting the optimized rank decomposition matrix, reducing the trainable parameter data of downstream tasks and making training more efficient.

[0162] After using LoRA to fine-tune the network weights of the initially trained self-attention model, a trained self-attention model is obtained. The trained self-attention model can be used to predict the health status through the above-mentioned prediction method, thereby obtaining the health status prediction result of the battery.

[0163] In the above implementation process, the network weights of the initially trained self-attention model are fine-tuned through a low-rank adaptive mechanism, which can reduce the impact of the powerful fitting ability of the self-attention model on the training accuracy, so that the trained self-attention model obtained after fine-tuning can more accurately predict the health status.

[0164] Based on the above embodiment, the specific fine-tuning method includes: obtaining a weight adjustment matrix corresponding to the weight matrix of the model target layer, wherein the model target layer is the layer whose parameters need to be fine-tuned in the self-attention model that has been initially trained, and then maintaining the element values ​​in the weight matrix unchanged, updating the element values ​​in the weight adjustment matrix to adjust the network weights of the model target layer, and obtaining the trained self-attention model after the adjustment is completed.

[0165] Among them, the structural diagram of fine-tuning using the low-rank adaptive mechanism can be shown as follows Figure 6 As shown, Figure 6 The input is x, x is a d-dimensional vector, and the output is h. During fine-tuning, the weight matrix in the self-attention model that has been initially trained can be divided into a frozen weight matrix W0 and a trainable weight adjustment matrix ΔW.

[0166] For the weight matrix W0 of a certain network layer in the original self-attention model, its output can be simply expressed as h=W0x. After introducing the weight adjustment matrix ΔW, the output can be expressed as h=(W0+ΔW)x=W0x+ΔWx.

[0167] If the number of parameters of ΔW is the same as W0, in order to reduce the number of parameters for training, ΔW can be decomposed into low rank, expressed as ΔW = BA, where B and A represent two low rank matrices, that is, the weight adjustment matrix contains two low rank matrices, so the weight adjustment matrix can be expressed as the product of these two low rank matrices. r<<d. Assuming d=1024,r=4,then the number of trainable parameters is 2×4×1024=8192, which is much smaller than the number of parameters of W0 1024×1024.

[0168] When updating the element values ​​in the weight adjustment matrix, the gradient of the model loss function can be used to update the element values ​​in the weight adjustment matrix, that is, the gradient of the function is used to perform reverse propagation in the network to update the element values ​​in the weight adjustment matrix instead of updating the network weights of the original model. Of course, in other implementations, the loss value can also be used directly for updating, such as adjusting the element values ​​in the weight adjustment matrix according to the size of the loss value.

[0169] In the above implementation process, the network weights of the model are updated by updating the element values ​​in the weight adjustment matrix, which can effectively reduce the amount of training of network parameters and improve training efficiency.

[0170] On the basis of the above embodiment, since the self-attention model has undergone preliminary training and has acquired a certain predictive ability, in order not to damage the original performance of the model, the weight adjustment matrix corresponding to the weight matrix of the target layer of the model can be initialized first to obtain the initial weight adjustment matrix.

[0171] The initialization here may refer to initializing the two low-rank matrices A and B in the weight adjustment matrix. The specific initialization values ​​can be obtained based on experience or a certain algorithm. In this way, when training the model, the algorithm can iterate from a reasonable starting point, reducing the problem of the model falling into a local optimal solution at the beginning of training.

[0172] Based on the above embodiment, if both low-rank matrices are randomly initialized, it may cause large changes in the output during the initial training of the model, which will damage the original performance of the model. Therefore, one of the two low-rank matrices can be initialized to 0, and the other low-rank matrix can be randomly initialized to obtain the initialized weight adjustment matrix.

[0173] For example, if ΔW is a randomly initialized parameter, directly adding the ΔW parameter to the weight matrix of the target layer of the model will damage the performance of the original model. To solve this problem, in this scheme, one of the two low-rank matrices is initialized to 0, for example, matrix B is initialized to 0, and matrix A can be initialized with random values, or matrix A can be initialized to 0 and matrix B to random values.

[0174] This is equivalent to applying the weight adjustment matrix to the model target layer in the self-attention model, and then the training sample data (the training sample data used here can be the data under the working conditions of the battery to be predicted) can be input into the self-attention model or the prediction model for training. During the training process, the original weight matrix is ​​frozen, and the element values ​​in the two low-rank matrices are updated to update the network weights of the model.

[0175] As training progresses, the matrix B will gradually provide additional information to reduce the training loss function loss:

[0176] h=(W0+ΔW)x=W0x st.ΔWx=0

[0177] Although ΔWx = 0 at the beginning of training, the gradient of loss to ΔW is not 0, that is:

[0178]

[0179] Therefore, ΔW can be updated during the training process, and this solution can use the gradient of the model loss function to update the element values ​​in the two low-rank matrices, so as to achieve the update of the network weights to achieve the model goal. It can be understood that if the self-attention model in the entire prediction model is fine-tuned, then during training, the network weights in the feature extraction model and the network weights of other network layers in the prediction model need to be solidified.

[0180] In order to reduce the impact of updating the dimension r in B and A on training, a scaling parameter α can be introduced to control the output of the network layer:

[0181]

[0182] In this way, the parameter α can be adjusted appropriately, which is roughly equivalent to adjusting the learning rate, thereby updating the element values ​​in matrix A and matrix B.

[0183] In the above implementation process, by initializing the weight adjustment matrix, and then updating the element values ​​in the two low-rank matrices instead of updating the network weights of the model, the amount of training of the network parameters can be effectively reduced and the training efficiency can be improved.

[0184] Based on the above embodiment, since the training accuracy is very sensitive to the parameter α and initialization, the variance of each training accuracy is large, and this unstable training will lead to sub-optimization of the model. In order to alleviate this problem, after each update of the element value in the weight adjustment matrix, the variance of the part of the output of the target layer of the model corresponding to the weight adjustment matrix can be controlled to a set value, which is determined according to the variance of the probability distribution obeyed by the input data.

[0185] It can be understood that the input data here can refer to the input data of the self-attention model or the weight adjustment matrix after initialization. If the variance of the probability distribution obeyed by the input data is Then the set value can be 1. If the variance of the probability distribution obeyed by the input data is N(0,2), then the set value can be 2. Therefore, the set value can be flexibly determined according to the variance of the probability distribution obeyed by the input data.

[0186] In the above implementation process, each time after updating the element value in the weight adjustment matrix, the variance of the part of the output of the model target layer corresponding to the weight adjustment matrix is ​​controlled within the set value, which can reduce the problem of model sub-optimization caused by the large difference in training accuracy each time.

[0187] On the basis of the above embodiment, the specific method of controlling the variance of the part of the output of the model target layer corresponding to the weight adjustment matrix within the set value may be: scaling the two low-rank matrices respectively so that the variance of the part of the output of the model target layer corresponding to the weight adjustment matrix is ​​adjusted to the set value, and then the two low-rank matrices after the scaling process can be used for the next element value update, that is, the next update using the gradient of the model loss function until the model training completion conditions are met.

[0188] It can be understood that during the model training process, the model needs to be trained multiple times, and the element values ​​in the two low-rank matrices are updated during each training. After each update of the element values ​​in the two low-rank matrices, the two low-rank matrices are scaled, and then the two low-rank matrices after the scaling are used for the next model training. That is to say, here, scaling the two low-rank matrices after each update of the element values ​​in the two low-rank matrices is performed after each update of the two low-rank matrices using the gradient of the model loss function during the training process, that is, the operation performed between the two updates of the element values ​​of the two low-rank matrices using the gradient of the model loss function twice. If the model training completion conditions are met (such as the number of training times reaches the set number of times or the loss value is less than the set value, etc.), the process can be stopped.

[0189] For example, suppose the weight adjustment matrix and the input of the model network layer are initialized to obey Then the part of the output of the model target layer corresponding to the weight adjustment matrix (y in the following formula can be understood as Figure 6 The right branch part of the graph can be expressed as:

[0190] Among them, x i represents the input of the network layer, w i Represents the element value of matrix A or matrix B.

[0191] Assuming that the input variables are independent and identically distributed, since Var(x1+x2)=Var(x1)+Var(x2), the variance of y is:

[0192]

[0193] To ensure that the variance of y is 1, the output can be explicitly normalized: That is, the variance of matrix A and matrix B can affect the variance of y, so scaling matrix A and matrix B is equivalent to adjusting the variance of matrix A and matrix B, and then adjusting the size of the variance of y so that the variance of y is adjusted to 1. Of course, in actual situations, the variance of y may not be absolutely 1, but may be around 1. The purpose of this solution is to adjust the variance of y to 1.

[0194] Therefore, when matrices A and B are scaled, for matrix A, the corresponding n is d, and for matrix B, the corresponding n is r, so d can be used as the scaling parameter of matrix A, and r can be used as the scaling parameter of matrix A.

[0195] Finally, the output of the network layer can be expressed as:

[0196]

[0197] In this way, B and A can be scaled directly by the dimensions r and d of matrices B and A, without adjusting the scaling parameter α, which can effectively reduce its impact on training accuracy.

[0198] It is understandable that in practical applications, there are other ways to control the variance of the part of the output of the model target layer corresponding to the weight adjustment matrix within the set value, which are not listed here one by one.

[0199] On the basis of the above embodiment, when the model target layer in the self-attention model is fine-tuned, the model target layer here can be all layers in the self-attention model that has been preliminarily trained, or it can also be a network layer in multiple self-attention units in the self-attention model that has been preliminarily trained. In this way, fine-tuning the network weights of all layers can take into account the overall prediction effect of the model, or fine-tuning the network weights of multiple self-attention units can take into account the training accuracy and training efficiency of the model.

[0200] When the model target layer includes all layers of the self-attention model, the network weights of all layers can be fine-tuned in the above manner, and each network weight is set with the above-mentioned weight adjustment matrix, so that multiple network weights can be fine-tuned synchronously.

[0201] In practice, it is found that fine-tuning only the network weights of the network layers in multiple self-attention units can significantly improve the prediction accuracy of the model, while fine-tuning other layers may lead to underfitting of the model, so the model target layer can only contain the network layers in multiple self-attention units.

[0202] It can be understood that multiple self-attention units include multiple network layers, and the most important of them is the Multi-Head Attention layer in the multi-head self-attention module. Fine-tuning the network weights of the Multi-Head Attention layer can not only improve the model accuracy, but also improve the training efficiency. Therefore, the model target layer can also refer to the Multi-Head Attention layer in the multi-head self-attention module.

[0203] In this case, if the network weights of the Multi-Head Attention layer are fine-tuned, the corresponding weight adjustment matrix may include: a weight matrix for a network layer used to combine the calculation results of multiple self-attention heads, a weight matrix for a network layer used to calculate the query matrix Q, a weight matrix for a network layer used to calculate the key matrix K, and a weight matrix for a network layer used to calculate the value matrix V. At least one of the following four weight adjustment matrices corresponds to the weight matrix.

[0204] The above query matrix Q can be expressed as Q = XW Q , the key matrix K can be expressed as K = XW K , the value matrix V can be expressed as V = XW V , the calculation formula for combining the calculation results of multiple self-attention heads can be expressed as MultiHead(Q,K,V)=Concat(head1,head2,…,head n )W O .

[0205] Among them, the weight matrices that can be fine-tuned include W Q ,W K ,W V ,W O At least one of these four, if these four weight matrices need to be fine-tuned, the above-mentioned low-rank adaptive mechanism can be used to fine-tune these four weight matrices separately, that is, each weight matrix initializes a corresponding weight adjustment matrix, so that the four weight adjustment matrices can be updated separately, and the four weight matrices can be updated separately.

[0206] In some embodiments, in order to improve the efficiency of fine-tuning, the same weight adjustment matrix can be initialized for the four weight matrices, so that one weight adjustment matrix can be updated to achieve the update of the four weight matrices. In this way, only the network weights of some network layers in the self-attention unit can be fine-tuned, because these network layers are the key layers in the self-attention model, and the self-attention mechanism is the key to affecting the prediction effect. Therefore, only fine-tuning these layers in the self-attention unit not only reduces the amount of calculation, but also reduces the problem of model overfitting, which can effectively improve the training accuracy of the model.

[0207] In conjunction with the above method embodiment, please refer to Figure 7 , Figure 7 This is a structural block diagram of a battery health status prediction device 300 provided in an embodiment of the present application. The device 300 may be a module, program segment or code on an electronic device. It should be understood that the device 300 is similar to the above-mentioned Figure 1 The method embodiment corresponds to and can be executed Figure 1 The various steps involved in the method embodiment and the specific functions of the device 300 can be found in the above description. To avoid repetition, the detailed description is appropriately omitted here.

[0208] Optionally, the device 300 includes:

[0209] A feature extraction module 310 is used to extract features from the battery operating status data through a feature extraction model in the prediction model to obtain status features;

[0210] The prediction module 320 is used to perform health state prediction according to the state characteristics through the self-attention model in the prediction model to obtain the health state prediction result of the battery.

[0211] Optionally, the operating status data of the battery includes data of multiple charge and discharge cycles;

[0212] The prediction module 320 is used to predict the health status according to the state characteristics corresponding to the data of each charge and discharge cycle through the self-attention model in the prediction model, so as to obtain the health status prediction result of each charge and discharge cycle.

[0213] Optionally, the operating status data of the battery includes data of multiple charge and discharge cycles; the prediction module 320 is used to reduce the value of at least one target element in the intermediate features obtained through the self-attention model in the prediction model to obtain a target feature, wherein the intermediate feature refers to the self-attention model that extracts the status feature, and each target element represents the correlation between the data of the current charge and discharge cycle corresponding to it and the data of a subsequent charge and discharge cycle; the health status prediction is performed based on the target feature through the self-attention model to obtain the health status prediction result of the battery.

[0214] Optionally, the prediction module 320 is used to set the value of at least one target element in the intermediate feature to 0 through the self-attention model to obtain a target feature.

[0215] Optionally, the prediction module 320 is used to apply a mask to the intermediate feature through the self-attention model to obtain a target feature, wherein the element value at a position in the mask corresponding to at least one target element in the intermediate feature is 0.

[0216] Optionally, the intermediate feature refers to a query matrix Q, a key matrix K or a value matrix V obtained by the self-attention model in a self-attention operation process;

[0217] Alternatively, the intermediate feature refers to a feature obtained based on at least two matrix operations among the query matrix Q, the key matrix K, and the value matrix V.

[0218] Optionally, the feature extraction model is a convolutional neural network model.

[0219] Please refer to Figure 8 , Figure 8 The structure block diagram of a model training device 400 provided in an embodiment of the present application, the device 400 may be a module, program segment or code on an electronic device. It should be understood that the device 400 is similar to the above Figure 5 The method embodiment corresponds to and can be executed Figure 5 The various steps involved in the method embodiment and the specific functions of the device 400 can be found in the above description. To avoid repetition, the detailed description is appropriately omitted here.

[0220] Optionally, the device 400 includes:

[0221] A model acquisition module 410 is used to acquire a self-attention model that has been initially trained;

[0222] The weight adjustment module 420 is used to fine-tune the network weights of the self-attention model that has been preliminarily trained through a low-rank adaptive mechanism to obtain a trained self-attention model.

[0223] Optionally, the weight adjustment module 420 is used to obtain a weight adjustment matrix corresponding to the weight matrix of a model target layer, wherein the model target layer is a layer whose parameters need to be fine-tuned in the self-attention model that has been initially trained; and to update the element values ​​in the weight adjustment matrix to adjust the network weights of the model target layer, and obtain a trained self-attention model after the adjustment is completed.

[0224] Optionally, the weight adjustment module 420 is used to initialize the weight adjustment matrix corresponding to the weight matrix of the target layer of the model to obtain the initialized weight adjustment matrix.

[0225] Optionally, the weight adjustment module 420 is used to initialize one of the two low-rank matrices in the weight adjustment matrix corresponding to the weight matrix of the model target layer to 0, and randomly initialize the other low-rank matrix to obtain the initialized weight adjustment matrix.

[0226] Optionally, the weight adjustment module 420 is also used to adjust the variance of the part of the output of the model target layer corresponding to the weight adjustment matrix to a set value after each update of the element value in the weight adjustment matrix, and the set value is determined based on the variance of the probability distribution obeyed by the input data.

[0227] Optionally, the weight adjustment matrix includes two low-rank matrices; the weight adjustment module 420 is used to scale the two low-rank matrices respectively so that the variance of the part of the output of the model target layer corresponding to the weight adjustment matrix is ​​adjusted to a set value.

[0228] Optionally, the model target layer includes a network layer of multiple self-attention units in the self-attention model that has been initially trained.

[0229] Optionally, the weight adjustment matrices corresponding to the network layers in the multiple self-attention units include: a weight adjustment matrix corresponding to at least one of the weight matrix of the network layer used to combine the calculation results of multiple self-attention heads, the weight matrix of the network layer used to calculate the query matrix, the weight matrix of the network layer used to calculate the key matrix, and the weight matrix of the network layer used to calculate the value matrix.

[0230] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0231] Please refer to Fig. 9 , Fig. 9A schematic diagram of the structure of an electronic device for executing a battery health status prediction method or a model training method provided in an embodiment of the present application, the electronic device may include: at least one processor 510, such as a CPU, at least one communication interface 520, at least one memory 530 and at least one communication bus 540. Among them, the communication bus 540 is used to realize direct connection and communication between these components. Among them, the communication interface 520 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The memory 530 can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 530 can optionally also be at least one storage device located away from the aforementioned processor. Computer-readable instructions are stored in the memory 530. When the computer-readable instructions are executed by the processor 510, the electronic device executes the above-mentioned Figure 1 or Figure 5 The method process is shown.

[0232] Understandably, Fig. 9 The structure shown is for illustration only, and the electronic device may also include Fig. 9 More or fewer components as shown, or with Fig. 9 Different configurations shown. Fig. 9 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0233] The present application embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the following is performed: Figure 1 or Figure 5 The method process in the method embodiment shown is executed by the electronic device.

[0234] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can perform the methods provided by the above method embodiments, for example, including:

[0235] The feature extraction model in the prediction model is used to extract the features of the battery operation status data to obtain the status features;

[0236] The health state prediction of the battery is performed according to the state characteristics through the self-attention model in the prediction model to obtain the health state prediction result of the battery.

[0237] In summary, the embodiments of the present application provide a battery health status prediction method, model training method and device. The method realizes status feature extraction through a feature extraction model with powerful feature extraction capabilities, which can better extract potential features in the battery's operating status data, and realizes health status prediction through a self-attention model. It can better pay attention to the important information in the status characteristics, and better capture the global information therein, so that more and more important information can be effectively utilized to realize prediction, thereby effectively improving the prediction accuracy.

[0238] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0239] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0240] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0241] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0242] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for predicting the health status of a battery, characterized in that: The method comprises: The feature extraction model in the prediction model is used to extract the features of the battery operation status data to obtain the status features; The health state prediction of the battery is performed according to the state characteristics through the self-attention model in the prediction model to obtain the health state prediction result of the battery.

2. The method according to claim 1, characterized in that The battery operation status data includes data of multiple charge and discharge cycles; The performing health state prediction according to the state feature by using the self-attention model in the prediction model to obtain the health state prediction result of the battery includes: The self-attention model in the prediction model is used to predict the health status according to the state characteristics corresponding to the data of each charge and discharge cycle, so as to obtain the health status prediction result of each charge and discharge cycle.

3. The method according to claim 1, characterized in that The operating status data of the battery includes data of multiple charge and discharge cycles; the self-attention model in the prediction model performs health status prediction according to the status characteristics to obtain the health status prediction result of the battery, including: The self-attention model in the prediction model reduces the value of at least one target element in the intermediate features obtained to obtain the target feature, wherein the intermediate feature refers to the state feature extracted by the self-attention model, and each target element represents the correlation between the data of the current charge and discharge cycle corresponding to it and the data of a subsequent charge and discharge cycle; The health state prediction of the battery is performed based on the target feature through the self-attention model to obtain a health state prediction result of the battery.

4. The method according to claim 3, characterized in that The step of reducing the value of at least one target element in the intermediate features obtained by the self-attention model in the prediction model to obtain the target feature comprises: The value of at least one target element in the intermediate feature is set to 0 through the self-attention model to obtain the target feature.

5. The method according to claim 4, characterized in that The step of setting the value of at least one target element in the intermediate feature to 0 by using the self-attention model to obtain the target feature includes: A mask is applied to the intermediate feature through the self-attention model to obtain a target feature, wherein the element value at a position in the mask corresponding to at least one target element in the intermediate feature is 0.

6. The method according to claim 3, characterized in that The intermediate features refer to the query matrix Q, key matrix K or value matrix V obtained by the self-attention model in the self-attention operation process; Alternatively, the intermediate feature refers to a feature obtained based on at least two matrix operations among the query matrix Q, the key matrix K, and the value matrix V.

7. The method according to any one of claims 1 to 6, characterized in that: The feature extraction model is a convolutional neural network model.

8. A model training method, characterized in that: The method comprises: Get the self-attention model that has been initially trained; The network weights of the initially trained self-attention model are fine-tuned through a low-rank adaptive mechanism to obtain a trained self-attention model.

9. The method according to claim 8, characterized in that The method of fine-tuning the network weights of the preliminarily trained self-attention model through a low-rank adaptive mechanism to obtain a trained self-attention model includes: Obtain a weight adjustment matrix corresponding to a weight matrix of a model target layer, wherein the model target layer is a layer whose parameters need to be fine-tuned in the self-attention model that has been preliminarily trained; The element values ​​in the weight adjustment matrix are updated to adjust the network weights of the target layer of the model, and a trained self-attention model is obtained after the adjustment is completed.

10. The method according to claim 9, characterized in that The step of obtaining a weight adjustment matrix corresponding to the weight matrix of the target layer of the model includes: Initialize the weight adjustment matrix corresponding to the weight matrix of the target layer of the model to obtain the initialized weight adjustment matrix.

11. The method according to claim 10, characterized in that The weight adjustment matrix includes two low-rank matrices; the weight adjustment matrix corresponding to the weight matrix of the target layer of the model is initialized to obtain the initialized weight adjustment matrix, including: One of the two low-rank matrices in the weight adjustment matrix corresponding to the weight matrix of the target layer of the model is initialized to 0, and the other low-rank matrix is ​​randomly initialized to obtain the initialized weight adjustment matrix.

12. The method according to claim 9, characterized in that The updating of the element values ​​in the weight adjustment matrix includes: After each update of the element values ​​in the weight adjustment matrix, the variance of the part of the output of the model target layer corresponding to the weight adjustment matrix is ​​adjusted to a set value, and the set value is determined based on the variance of the probability distribution obeyed by the input data.

13. The method according to claim 12, characterized in that The weight adjustment matrix includes two low-rank matrices; and adjusting the variance of the part of the output of the target layer of the model corresponding to the weight adjustment matrix to a set value includes: The two low-rank matrices are scaled respectively so that the variance of the part of the output of the model target layer corresponding to the weight adjustment matrix is ​​adjusted to a set value.

14. The method according to claim 9, characterized in that The model target layer includes a network layer in a plurality of self-attention units in the self-attention model that has been preliminarily trained.

15. The method according to claim 14, characterized in that The weight adjustment matrices corresponding to the network layers in the multiple self-attention units include: a weight adjustment matrix corresponding to at least one of the weight matrix of the network layer used to combine the calculation results of multiple self-attention heads, the weight matrix of the network layer used to calculate the query matrix, the weight matrix of the network layer used to calculate the key matrix, and the weight matrix of the network layer used to calculate the value matrix.

16. A battery health status prediction device, characterized in that: The device comprises: A feature extraction module is used to extract features from the battery operation status data through a feature extraction model in the prediction model to obtain status features; A prediction module is used to predict the health state of the battery according to the state characteristics through a self-attention model in the prediction model to obtain a health state prediction result of the battery.

17. A model training device, characterized in that: The device comprises: Model acquisition module, used to obtain the self-attention model that has been initially trained; The weight adjustment module is used to fine-tune the network weights of the self-attention model that has been preliminarily trained through a low-rank adaptive mechanism to obtain a trained self-attention model.

18. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 15 is executed.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 15 is performed.

Images