Language Model-Based Battery Health Status Prediction Methods, Devices, and Products

Abstract

This invention provides a method, apparatus, and product for predicting battery health status based on a language model. Utilizing the above technical solution, data on the electrical parameters of the target battery changing over time is acquired on the application side and processed to obtain input data that conforms to the input features of the language model. The battery health status corresponding to the target battery is obtained through the output features of the target language model. On the training side, the physical principles of the battery can be used as physical constraints to freeze the pre-trained model backbone in the basic language model while training the adjustable components, thereby obtaining the target language model. Thus, by leveraging the generalization ability of the pre-trained model backbone and the generalization ability brought about by the interpretability of physical constraints, the amount of training data required during model training is reduced, the modeling process is optimized, and modeling efficiency is improved, thereby enhancing the effectiveness of battery health status prediction.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method, apparatus and product for predicting battery health status based on a language model. Background Technology

[0002] During battery application, health status estimation is necessary to assess the degree of battery degradation, thereby determining battery performance, lifespan, and safety. Battery health status cannot be directly measured and must be estimated using raw data. A feasible data-driven health status estimation method requires data from the entire battery lifecycle to establish a mapping relationship. However, this method requires excessive data volume and a long acquisition cycle, resulting in low data acquisition efficiency. Furthermore, the established model may not be applicable to current battery technology development. In summary, current battery health status prediction methods are time-consuming, require large amounts of data, leading to high modeling difficulty and low model effectiveness. Therefore, a language model-based battery health status prediction method and corresponding model training method can be designed to simplify the required data volume, optimize the modeling process, improve modeling efficiency, and thus enhance the effectiveness of battery health status prediction. Summary of the Invention

[0003] The embodiments of the present invention provide a method, device and product for predicting battery health status based on a language model, in order to solve the problems of long modeling time, large data volume and high modeling difficulty in current battery health status prediction methods, and low effectiveness of the obtained models.

[0004] To achieve the above objectives, in a first aspect, this application provides a battery health state prediction method based on a language model, the method comprising:

[0005] Determine the target battery data;

[0006] The target battery data includes the change of at least one electrical parameter of the target battery over time;

[0007] The target battery data is processed to obtain a first input feature corresponding to the input form of the target language model;

[0008] The target language model is obtained by training a base language model through cue learning. The base language model includes a pre-trained model backbone, adjustable components, and a cue. The pre-trained model backbone includes a GPT-2 model backbone. The adjustable components include an input translation layer, an output mapping layer, and a feature reconstruction layer. The cue is used to generate cue words, which are used to indicate the second input features of the model backbone.

[0009] Furthermore, the input translation layer is used to obtain the corresponding text representation sequence features based on the input features corresponding to the battery training data; the feature reconstruction layer is used to obtain the complete output charging curve based on the output features of the target language model; the complete output charging curve is used to optimize the prompt words corresponding to the prompt learning process; and the output mapping layer is used to obtain the battery health status of the corresponding battery based on the output features of the target language model.

[0010] The first input feature and the second input feature are input into the target language model to obtain the battery health status corresponding to the target battery;

[0011] And, the cue learning includes:

[0012] Determine the battery training data and input the battery training data into the base language model;

[0013] The battery training data includes the input charging curves for each battery.

[0014] Determine the physical constraints;

[0015] The physical constraints are used to indicate the constraints on the physical model and electrical parameters of the target battery.

[0016] Based on the aforementioned physical constraints, determine the optimization objective function;

[0017] Based on the optimization objective function, generate the output charging curve;

[0018] During the training phase, the component structure and component features of each of the adjustable components and the prompter are adjusted according to the output charging curve; and during the testing phase, the component structure and component features of the prompter are adjusted according to the output charging curve.

[0019] As an optional implementation, determining the physical constraints includes:

[0020] Based on the first-order RC equivalent circuit model, the first formula is determined;

[0021] The first formula is used to indicate the electrical behavior of the first-order RC equivalent circuit model, and the first formula includes:

[0022]

[0023] Where u is the terminal voltage, I is the equivalent current, and θ1(T) and θ2(T) are temperature-related functions used to indicate the electrical behavior of the first-order RC equivalent circuit model;

[0024] And, determining the optimization objective function based on the physical constraints includes:

[0025] Determine the unsupervised loss function based on the second formula;

[0026] The second formula includes:

[0027]

[0028] The objective function is used to indicate the second norm based on the physical constraints, formed by the output values ​​and corresponding input values ​​of each sequence on the output charging curve, and the optimization condition of minimizing the sum of the products of the first optimization coefficient λ and the second norm based on the first formula and the physical constraints.

[0029] As an optional implementation, generating the output charging curve based on the optimized objective function includes:

[0030] Based on the battery training data, determine the encoding sequence;

[0031] The preset data in the encoded sequence is masked to generate target encoded data;

[0032] Based on the optimization objective function, the target encoded data is expanded to generate an output charging curve.

[0033] As an optional implementation, the first input feature includes sequence words, the battery training data includes an input charging curve, and the step of determining the battery training data and inputting the battery training data into the base language model includes:

[0034] Obtain the input charging curves from the target battery dataset, and determine the corresponding time series data based on each input charging curve;

[0035] Based on the time series data, an input vector is determined, and the corresponding text representation sequence features are determined by the input translation layer using a multi-head attention mechanism.

[0036] Based on the text representation sequence features, sequence words are determined, and the sequence words are input into the model backbone of the basic language model.

[0037] As an optional implementation, the method further includes:

[0038] The input charging curve is constrained by the physical constraints, and the component structure and component features of the input translation layer are obtained by training based on the constrained input charging curve and backpropagation features.

[0039] The propagation path of the backpropagation features includes: from the output mapping layer to the output feature layer of the model backbone to the main body of the model backbone to the input translation layer;

[0040] The equivalent circuit model is used to constrain the complete output charging curve of the feature reconstruction layer. Based on the constrained complete output charging curve, prompt words are trained and obtained. The prompt words and the sequence words are input into the model backbone of the basic language model to obtain the output features of the current model backbone. Based on the output features, the component structure and component features of the feature reconstruction layer and the output mapping layer are trained and obtained.

[0041] Secondly, this application provides a battery health state prediction device based on a language model, the device comprising:

[0042] The determination module is used to determine the target battery data;

[0043] The target battery data includes the change of at least one electrical parameter of the target battery over time;

[0044] The processing module is used to process the target battery data to obtain a first input feature corresponding to the input form of the target language model;

[0045] The target language model is obtained by training a base language model through cue learning. The base language model includes a pre-trained model backbone, adjustable components, and a cue. The pre-trained model backbone includes a GPT-2 model backbone. The adjustable components include an input translation layer, an output mapping layer, and a feature reconstruction layer. The cue is used to generate cue words, which are used to indicate the second input features of the model backbone.

[0046] Furthermore, the input translation layer is used to obtain the corresponding text representation sequence features based on the input features corresponding to the battery training data; the feature reconstruction layer is used to obtain the complete output charging curve based on the output features of the target language model; the complete output charging curve is used to optimize the prompt words corresponding to the prompt learning process; and the output mapping layer is used to obtain the battery health status of the corresponding battery based on the output features of the target language model.

[0047] The processing module is further configured to input the first input feature and the second input feature into the target language model to obtain the battery health status corresponding to the target battery;

[0048] And, the specific methods by which the processing module performs prompting learning include:

[0049] Determine the battery training data and input the battery training data into the base language model;

[0050] The battery training data includes the input charging curves for each battery.

[0051] Determine the physical constraints;

[0052] The physical constraints are used to indicate the constraints on the physical model and electrical parameters of the target battery.

[0053] Based on the aforementioned physical constraints, determine the optimization objective function;

[0054] Based on the optimization objective function, generate the output charging curve;

[0055] During the training phase, the component structure and component features of each of the adjustable components and the prompter are adjusted according to the output charging curve; and during the testing phase, the component structure and component features of the prompter are adjusted according to the output charging curve.

[0056] Thirdly, this application provides an electronic device, characterized in that the electronic device includes a memory and a processor, the memory storing at least one program, the at least one program being executed by the processor to implement the method as described in the first aspect.

[0057] Fourthly, this application provides a computer-readable storage medium storing at least one program that is executed by a processor to implement the method described in the first aspect.

[0058] Fifthly, this application provides a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the method described in the first aspect.

[0059] The battery health status prediction method based on a language model provided in this application obtains data on the electrical parameters of the target battery changing over time on the application side, processes the data to obtain input feature form conforming to the language model, and uses this data as input data for the target language model. This input data is then fed into the target language model, which is trained through physical constraint prompts. The battery health status corresponding to the target battery is obtained through the output features of the target language model. On the training side, the physical principles of the battery are used as physical constraints to freeze the pre-trained model backbone in the basic language model, while training the adjustable components to obtain the target language model. Thus, by utilizing the generalization ability of the pre-trained model backbone and the generalization ability brought by the interpretability of physical constraints, the amount of training data required during model training is reduced. The corresponding output features are obtained through the language model with physical constraints, and then a complete prediction curve or battery health status is generated from the output features. This simplifies the required amount of data, optimizes the modeling process, improves modeling efficiency, and thus enhances the effectiveness of battery health status prediction. Attached Figure Description

[0060] Figure 1 A flowchart illustrating a language model-based battery health state prediction method provided in this application embodiment;

[0061] Figure 2 This is a schematic diagram of the structure of a target language model provided in an embodiment of this application;

[0062] Figure 3 This is a schematic diagram of an equivalent circuit model provided in an embodiment of this application;

[0063] Figure 4 This is a schematic diagram illustrating the effect of a language model-based battery health state prediction method provided in an embodiment of this application. Detailed Implementation

[0064] A key direction in battery management systems is State of Health (SOH) estimation, which cannot be directly measured and can only be estimated to assess the degree of battery degradation. Accurate SOH estimation is crucial for safe and efficient energy management and the development of next-generation batteries. The rapid development of lithium-ion batteries (LIBs) has led to their widespread application in various fields. Taking LIBs as an example, existing data-driven SOH estimation methods require lifetime data to establish a mapping relationship for each LIB. Due to the extremely long lifespan of modern lithium-ion batteries, obtaining lifetime training data necessitates extensive discharge aging experiments, which often take months to years, significantly hindering their application in battery management. This also greatly impedes the development of battery management systems (BMS) and new battery technologies. New SOH estimation methods based on transfer learning alleviate the burden of data collection to some extent; however, their assumptions about the target LIB dataset are far from perfect. For example, some methods require labeling 25% of the target LIB dataset, while others require features of the unlabeled target LIB data. In such cases, obtaining the target LIB dataset still requires considerable effort.

[0065] In contrast, this application assumes that only one sample of the target LIB is accessed at a time, which is more realistic. For example, when a new mobile phone is charged once a day, the BMS can only collect one unlabeled charging characteristic curve. It would be meaningful if this application could use health information from existing samples to update the model, rather than spending a lot of time and effort collecting large amounts of data for updates.

[0066] Beyond its tremendous success in language modeling, foundational models have also achieved great success in fields such as computer vision, time series, audio, and graphics. Foundational models are generally defined as models trained on a wide range of data, capable of adapting to a variety of downstream tasks. For general time series data, such as battery charging data, there are two typical approaches: collect enough data to train a large model from scratch, or explore the feasibility of pre-trained large language models (LLMs), i.e., reprogramming the input types and / or fine-tuning the LLM.

[0067] Because the time required to collect datasets cannot keep pace with the development of new battery technologies, the practice of collecting large datasets and training a large LIB model is unsustainable. Therefore, this application proposes a target language model leveraging the powerful generalization capabilities of LLM. This application designs a translation layer to reprogram battery charging data into a text prototype representation more suitable for the language model's functionality, unifying data across different patterns and adapting LLM to battery tasks. To further enhance the model's generalization ability, this application proposes a novel Physically Guided Test Time Cue Adjustment (PGTPT) strategy, which fully explores health information in a single test by learning adaptive cues on the fly. PGTPT optimizes cues by guiding the LLM to generate a complete battery charging curve. This strategy also effectively mitigates the cumulative error caused by time distribution shifts due to changes in battery internal mechanisms. GPT4Battery achieves state-of-the-art accuracy on a wide range of benchmarks without requiring additional training data or annotations.

[0068] It should be noted that, in the implementation embodiments of this application, GPT4Battery, i.e., the model architecture corresponding to this application, should be understood as a form of implementation of a target language model.

[0069] Taking LIB state estimation as an example, the following explains the motivation for LLM (Large Language Model) driven and physically guided test-time learning.

[0070] State of Health (SOH) is an indicator for assessing the degree of battery degradation. Although definitions vary, it is generally accepted that the ratio of current capacity to initial capacity has a significant impact on battery management tasks, such as driving range estimation and lifespan prediction. This work will also focus on defining capacity as a health indicator.

[0071]

[0072] Here, Qnominal is the nominal capacity of a brand-new battery, while Qcycle is the capacity after a full charge in the current cycle. However, capacity measurement requires fully charging or discharging the battery according to a specific procedure, which is impractical for batteries in use. This has spurred research into estimating SOH using partial charge curves.

[0073] In the field of battery research, there are two main methods for estimating battery state: modeling-based methods and data-driven methods. Modeling-based methods involve building a simulation model comparable to a lithium-ion battery (LIB) and using parameter estimation to identify the state of a real battery. The two most studied models are the equivalent circuit model (ECM) and the pseudo-two-dimensional (P2D) model. These models help to gain a deeper understanding of the internal dynamics of the battery, such as lithium-ion diffusion and electrochemical kinetics. While these models provide a detailed explanation of how lithium-ion batteries work, a trade-off must be made between accuracy and computational efficiency in online applications, as detailed models lead to complex partial differential equations, and solving PDEs requires significant memory and computational power. In contrast, data-driven methods offer both high accuracy and extremely short inference times. However, data-driven methods still have two drawbacks: their generalization ability is heavily dependent on the data and they lack interpretability. First, unlike general time-series data, acquiring battery data requires extremely expensive degradation experiments, often taking months to years, making the collection of training data a time-consuming and resource-intensive task. Second, because usage often spans months to years, the internal mechanisms of the battery frequently change, such as side reactions and the stability of the solid electrolyte interface. Therefore, battery data is likely to involve changes in time distribution, which makes purely data-driven models less accurate in the mid-to-late stages of the battery lifecycle.

[0074] Specifically, to overcome the limitations of purely data-driven methods, it is essential to establish a model with high generality and interpretability. Recent research has evaluated the excellent performance of LLM (Language Pre-training) on ​​zero-point time series tasks, inspiring this application to leverage the generalization capabilities of LLM to save on tedious data collection. Regarding interpretability, several fusion architectures of physics-based and machine learning have been investigated in Physical Information Machine Learning (PIML) techniques. These methods combine physical models, such as Neumann's pseudo-two-dimensional (P2D), with basic neural networks as an unsupervised loss or to generate training data for machine learning models using physical models. To better combine the generality of LLM with the interpretability of physics-based models, this application aims to embed knowledge and rules from physics-based models into LLM in the form of prompts. Furthermore, few previous works have emphasized the inherent health information in the partial charging curves of test samples, reflecting the latest and most realistic battery state. If properly guided by physical rules, this sample can effectively supplement data shortages.

[0075] Therefore, this application designs a Physically Guided Test Time Cue Adjustment (PGTPT) algorithm, which learns adaptive cues using a single test sample. By guiding an LLM to generate a complete battery charging curve to optimize the cues, the generalization ability of the model is enhanced, thus compensating for the lack of interpretability in purely data-driven models.

[0076] This application proposes a target language model, also referred to as GPT4Battery in subsequent embodiments, which, compared to existing technologies in the art, serves as a foundational model for cross-battery SOH estimation, alleviating the burden of months to years of degradation experiments in data collection. This application leverages a single test sample to learn adaptive cues on the fly, thereby enhancing the model's generalization and adaptability in real-world scenarios. Extensive experiments were conducted on five widely recognized LIB datasets. GPT4Battery achieves state-of-the-art accuracy in cross-battery SOH estimation, comparable to domain adaptation or fine-tuning methods that require additional training data. Specific implementation details will be provided.

[0077] The technical concept of this application lies in acquiring data on the changes of electrical parameters of the target battery over time on the application side, processing the data to obtain input feature forms that conform to the language model, and using this data as input data for the target language model. This input data is then fed into the target language model, which is trained through physical constraint prompts. The output features of the target language model yield the battery health status corresponding to the target battery. On the training side, the physical principles of the battery are used as physical constraints to freeze the pre-trained model backbone in the basic language model, while training the adjustable components to obtain the target language model. This approach leverages the generalization ability of the pre-trained model backbone and the generalization ability brought about by the interpretability of physical constraints, reducing the amount of training data required during model training. The corresponding output features are obtained through the language model with physical constraints, and then a complete prediction curve or battery health status is generated from these output features. This simplifies the required data volume, optimizes the modeling process, improves modeling efficiency, and thus enhances the effectiveness of battery health status prediction.

[0078] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0079] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0080] Example 1:

[0081] Please see Figure 1 , Figure 1 This is a flowchart illustrating a language model-based battery health state prediction method provided in an embodiment of this application. Figure 1 As shown, this application provides a battery health state prediction method based on a language model, the method comprising:

[0082] Determine the target battery data;

[0083] The target battery data includes the change of at least one electrical parameter of the target battery over time;

[0084] The target battery data is processed to obtain a first input feature corresponding to the input form of the target language model;

[0085] The target language model is obtained by training a base language model through cue learning. The base language model includes a pre-trained model backbone, adjustable components, and a cue. The pre-trained model backbone includes a GPT-2 model backbone. The adjustable components include an input translation layer, an output mapping layer, and a feature reconstruction layer. The cue is used to generate cue words, which are used to indicate the second input features of the model backbone.

[0086] Furthermore, the input translation layer is used to obtain the corresponding text representation sequence features based on the input features corresponding to the battery training data; the feature reconstruction layer is used to obtain the complete output charging curve based on the output features of the target language model; the complete output charging curve is used to optimize the prompt words corresponding to the prompt learning process; and the output mapping layer is used to obtain the battery health status of the corresponding battery based on the output features of the target language model.

[0087] The first input feature and the second input feature are input into the target language model to obtain the battery health status corresponding to the target battery.

[0088] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of a target language model provided in an embodiment of this application, such as... Figure 2 As shown, as an optional implementation, the adjustable component includes an input translation layer, an output mapping layer, and a feature reconstruction layer;

[0089] The input translation layer is used to obtain the corresponding text representation sequence features based on the input features corresponding to the battery training data. The feature reconstruction layer is used to obtain the complete output charging curve based on the output features of the target language model. The complete output charging curve is used to optimize the prompt words corresponding to the prompt learning process. The output mapping layer is used to obtain the battery health status of the corresponding battery based on the output features of the target language model.

[0090] The model corresponding to the method in this application includes an input translation layer, an output mapping layer, a feature reconstruction layer, and a pre-trained frozen LLM structure, i.e., four main parts of the model backbone. The training process only adjusts three adjustable components without optimizing the specific features of the pre-trained model backbone. During training, all three adjustable components are jointly trained with the prompter. During testing, the complete battery charging curve is reconstructed by guiding the LLM, and only the prompts are optimized. Specific details can be found in the descriptions of the various implementation methods.

[0091] By designing three adjustable components—an input translation layer, an output mapping layer, and a feature reconstruction layer—the pre-trained model backbone features are no longer adjusted during actual training. Instead, each adjustable component is trained to obtain more accurate output features, thereby improving the accuracy of health status prediction. Leveraging the generalization ability of the pre-trained model backbone and the generalization ability brought by the interpretability of physical constraints, the amount of training data required during model training is reduced. The corresponding output features are obtained through a language model of physical constraints, and then a complete prediction curve or battery health status is generated from the output features. This simplifies the required amount of data, optimizes the modeling process, improves modeling efficiency, and ultimately enhances the effectiveness of battery health status prediction.

[0092] And, the cue learning includes:

[0093] Determine the battery training data and input the battery training data into the base language model;

[0094] The battery training data includes the input charging curves for each battery.

[0095] Determine the physical constraints;

[0096] The physical constraints are used to indicate the constraints on the physical model and electrical parameters of the target battery.

[0097] Based on the aforementioned physical constraints, determine the optimization objective function;

[0098] Based on the optimization objective function, generate the output charging curve;

[0099] During the training phase, the component structure and component features of each of the adjustable components and the prompter are adjusted according to the output charging curve; and during the testing phase, the component structure and component features of the prompter are adjusted according to the output charging curve.

[0100] In summary, corresponding strategies were designed on both the training and application sides. On the application side, data on the electrical parameters of the target battery changing over time was acquired and processed to obtain data in the form of input features conforming to the language model. This data was then used as input data for the target language model, which was trained using physical constraints. The output features of the target language model were used to obtain the battery health status corresponding to the target battery. On the training side, the physical principles of the battery were used as physical constraints to freeze the pre-trained model backbone in the basic language model, while the adjustable components were trained to obtain the target language model. Thus, by utilizing the generalization ability of the pre-trained model backbone and the generalization ability brought by the interpretability of physical constraints, the amount of training data required during model training was reduced. The corresponding output features were obtained through the language model with physical constraints, and then a complete prediction curve or battery health status was generated from the output features. This simplified the amount of data required, optimized the modeling process, improved modeling efficiency, and thus improved the effectiveness of battery health status prediction.

[0101] As an optional implementation, the battery training data includes an input charging curve. The step of determining the battery training data and inputting the battery training data into the base language model includes:

[0102] Obtain the input charging curves from the target battery dataset, and determine the corresponding time series data based on each input charging curve;

[0103] Based on the time series data, the corresponding text representation sequence features are determined through the input translation layer;

[0104] Based on the text representation sequence features, sequence words are determined, and the sequence words are input into the model backbone of the basic language model.

[0105] As an optional implementation, the first input feature includes sequence words, the battery training data includes an input charging curve, and the step of determining the battery training data and inputting the battery training data into the base language model includes:

[0106] Obtain the input charging curves from the target battery dataset, and determine the corresponding time series data based on each input charging curve;

[0107] Based on the time series data, an input vector is determined, and the corresponding text representation sequence features are determined by the input translation layer using a multi-head attention mechanism.

[0108] Based on the text representation sequence features, sequence words are determined, and the sequence words are input into the model backbone of the basic language model.

[0109] Specifically, this application formalizes the cross-cell SOH estimation problem. Given a well-collected battery dataset, X∈R, representing a partial constant current input charging curve with time step T, and y representing the SOH label. This application includes samples of the battery lifetime consisting of S labels, i.e., S={(x1,y1),(x2,y2),...,(xS,yS)}. For a new battery, this application sequentially obtains target features through daily charging and discharging, denoted as T={(x1,y1),x2,...,xT}. The goal of this application is to estimate each target yt, where y1 can be considered as a new battery with SOH=100%.

[0110] This application uses the input voltage sampling window ΔV as univariate time series data. During constant current charging, the LIB charge is obtained by integrating the current at each time step. This application extends the current into a d-dimensional vector and normalizes it using the nominal capacity Qnominal of the LIB to adapt it to different LIBs. Then, this application represents this series of data as X∈R^1×T. Furthermore, consecutive timestamps can be aggregated into overlapping patch markers of length Lp, so the total number of blocks in the input sequence is... Where S represents the horizontal sliding step size, and T is the number of samples on the aforementioned charge-discharge curve. Given these patches, this application can obtain the patch embedding XP∈R^P×D, where D is the dimension of the word embedding, which facilitates subsequent translation.

[0111] This application now obtains the patch embedding form of the input sequence. To align modal consistency between time-series data and language, and to activate the time-series understanding capability of LLM, this application reprograms the patch embeddings into the pre-trained word embedding space of LLM. However, for the word embedding space of LLM, E∈R^V×D, where V is the vocabulary size, the vocabulary size in large language models can be very large. Directly utilizing E would result in a huge and potentially dense reprogramming space. To improve computational efficiency and increase the difficulty of capturing relevant source tags, this application uses a linear representation as E'∈R^V'×D, where V'<<V, thereby reducing the amount of data. Then, this application uses a multi-head cross-attention layer composed of a query matrix Q, a keyword matrix K, and a value matrix V to align the time-series patches and text prototypes. In this way, the text prototypes can learn cues in the language, which can represent relevant local patch information, allowing each attention head to capture different relevant features.

[0112]

[0113] Thus, this application has successfully re-encoded the input battery data into a sequence embedding, effectively adapting the modalities of time series data and natural language to fully utilize the capabilities of LLM.

[0114] By regularizing and segmenting the input charging curve, corresponding time-series data can be obtained. Through the multi-head attention mechanism of the input translation layer, combined with curve and cue word features, the corresponding text representation sequence features are determined. Based on these text representation sequence features, sequence words are identified and input into the backbone of the basic language model to execute subsequent output and training processes. This leverages the generalization ability of the pre-trained model backbone and the generalization ability brought by the interpretability of physical constraints, reducing the amount of training data required during model training. The corresponding output features are obtained through the language model with physical constraints, and then a complete prediction curve or battery health status is generated from these output features. This simplifies the required data volume, optimizes the modeling process, improves modeling efficiency, and ultimately enhances the effectiveness of battery health status prediction.

[0115] Please see Figure 3 , Figure 3 This is a schematic diagram of an equivalent circuit model provided in an embodiment of this application. Figure 3 As shown, as an optional implementation, determining the physical constraints includes:

[0116] Based on the first-order RC equivalent circuit model, the first formula is determined;

[0117] The first formula is used to indicate the electrical behavior of the first-order RC equivalent circuit model, and the first formula includes:

[0118]

[0119] Where u is the terminal voltage, I is the equivalent current, and θ1(T) and θ2(T) are temperature-related functions used to indicate the electrical behavior of the first-order RC equivalent circuit model;

[0120] And, determining the optimization objective function based on the physical constraints includes:

[0121] Determine the unsupervised loss function based on the second formula;

[0122] The second formula includes:

[0123]

[0124] The objective function is used to indicate the second norm based on the physical constraints, formed by the output values ​​and corresponding input values ​​of each sequence on the output charging curve, and the optimization condition of minimizing the sum of the products of the first optimization coefficient λ and the second norm based on the first formula and the physical constraints.

[0125] For details, please refer to the relevant descriptions later in this application. The electrical behavior of the equivalent circuit model is determined by the first formula, which serves as a constraint on the physical conditions of the battery. A corresponding loss function is established by using the second norm corresponding to the electrical behavior and the second norm related to the test data. Based on the loss function, a corresponding output charging curve can be generated for model optimization and prompt word training, making the model physically interpretable. This also enables unsupervised learning with partial data masking, as detailed in the following implementation methods.

[0126] Please see Figure 4 , Figure 4 This illustration shows the effect of a language model-based battery health state prediction method provided in this application embodiment. As an optional implementation, generating an output charging curve based on the optimized objective function includes:

[0127] Based on the battery training data, determine the encoding sequence;

[0128] The preset data in the encoded sequence is masked to generate target encoded data;

[0129] Based on the optimization objective function, the target encoded data is expanded to generate an output charging curve.

[0130] Therefore, by masking part of the data and guiding the missing data to generate a complete output charging curve, and by making the model physically interpretable, the amount of data required is reduced, the level of automation of training is improved, and the difficulty of data acquisition is reduced through unsupervised learning or partial self-supervised learning.

[0131] As an optional implementation, the physical constraints include an equivalent circuit model. The step of training the target language model by prompting learning on each of the adjustable components of the basic language model using the physical constraints to obtain the target language model includes:

[0132] The input charging curve is constrained by the equivalent circuit model, and the component structure and component features of the input translation layer are obtained by training based on the constrained input charging curve and backpropagation features.

[0133] The propagation path of the backpropagation features includes: from the output mapping layer to the output feature layer of the model backbone to the main body of the model backbone to the input translation layer;

[0134] The equivalent circuit model is used to constrain the complete output charging curve of the feature reconstruction layer. Based on the constrained complete output charging curve, prompt words are trained and obtained. The prompt words and the sequence words are then input into the model backbone of the basic language model to obtain the output features of the current model backbone. Based on the output features, the component structure and component features of the feature reconstruction layer and the output mapping layer are trained and obtained.

[0135] Specifically, in real-world battery usage scenarios, some feasible implementations obtain only one test sample at a time. Generating a corresponding curve to describe the battery's health characteristics using this sample would be a process that could take years. This application guides the LLM to adapt to the downstream battery SOH estimation task by re-encoding the input into a textual prototype representation and a linear output projection. Specifically, based on the needs of actual application scenarios and to reduce the cumulative error caused by changes in time distribution, this application uses a Physical Model-Guided Test Time Cue Learning (PGTPT) method. This method optimizes the language model's cues by guiding the LLM to generate a complete battery charging curve. See the relevant implementation details for further information.

[0136] Compared to direct fine-tuning of the model, cue learning only requires modifying the context of the model's input to learn beyond the pre-trained model. Therefore, this technique does not distort pre-trained features or introduce domain-specific behavior, thus avoiding the loss of LLM's out-of-distribution generalization ability.

[0137] Test-time training was originally proposed as a strategy to improve the generalization ability of models under distribution shifts. This application improves the generalization ability of LLM across battery tasks by optimizing hints at test time.

[0138]

[0139] F represents the model, and it's important to emphasize that no additional labels or data are needed besides the zero-point test samples. This means it's an unsupervised learning objective. Now, let's begin by introducing the ECM model.

[0140] The equivalent circuit model (ECM) is a widely used model to describe the electrical behavior of a battery in terms of voltage, current, resistance, and capacitance. The first-order Thevenin model is considered to accurately and adequately simulate the battery's state and is characterized by its simplicity and high computational efficiency. In this model, the battery's electrical behavior can be represented by a voltage source, with its internal ohmic resistance denoted as R. The parallel branch shown in the diagram is used to simulate the battery's polarization effect. u and I represent the terminal voltage and current that can be collected during use, respectively. According to Kirchhoff's laws, the battery's electrical behavior can be characterized by the following physical equations:

[0141] uOCV=uR+up+u

[0142]

[0143] The circuit parameters can be found by reference. Figure 4 The relevant locations are shown.

[0144] According to recent research, these coefficients can be written as a function of temperature, i.e.:

[0145]

[0146] This means that the terminal voltage u, current I, and temperature should, in principle, follow the physical state differential equation (ODE) function of the battery. The equivalent circuit model (ECM) has good qualities, such as ease of implementation and relatively accurate description of battery characteristics, which prompted this application to use it as a physical constraint.

[0147] Since labels cannot be adjusted during testing, this application employs an unsupervised loss method for timely adjustment. In battery research, a complete charge / discharge curve from the lower voltage limit to the upper voltage limit can describe the aging pattern of a lithium-ion battery (LIB), thus theoretically allowing for the determination of the battery's accurate health status. Inspired by this, this application aims to generate a complete battery charging curve from a partially obscured charging curve by guiding a pre-trained language model (LLM) during the design phase. This curve can then serve as test-time training (TTT) for self-supervised learning. Specifically, this application inputs a test sample... Then re-encoded into a token representation This application randomly masks a portion of the markers, in the form MaskrZ(i). The aim of this application is to utilize these masked portions to generate a complete battery charging curve. Where T′>T. However, without any annotation of the complete curve, it is rare to generate a meaningful curve. Now, this application can utilize the physical laws in the equivalent circuit model (ECM) to guide a pre-trained language model (LLM) to generate charging curves that conform to the physical laws.

[0148] In summary, the optimization objective of this application can be written as:

[0149]

[0150] In this application, GPT-2 can be used as the pre-trained large-scale language model (LLM) backbone. During training on the source dataset, the reconstruction layer, output layer, and all labeled embeddings are trained simultaneously. It is worth noting that when updating the output linear layer, this application uses the target lithium-ion battery (LIB) features (SOH = 100%) from the first cycle. When transferring to the target LIB dataset, this application also uses the features from the first cycle (SOH = 100%) to update the output linear layer. During testing, this application uses a single test sample to further optimize the cue labels to guide the LLM reconstruction of a physically accurate complete battery charging curve, i.e., the complete battery charging curve of the 1-RC ECM model.

[0151] By using an equivalent circuit model to explain the physical mechanism of the battery, relevant parameters are constrained, and the component structure and characteristics of each adjustable component are obtained, thus yielding more accurate output features and improving the accuracy of battery health status prediction. Utilizing the generalization ability of the pre-trained model backbone and the generalization ability brought by the interpretability of physical constraints reduces the amount of training data required during model training. Corresponding output features are obtained through a language model of physical constraints, and then a complete prediction curve or battery health status is generated from these output features. This simplifies the required amount of data, optimizes the modeling process, improves modeling efficiency, and ultimately enhances the effectiveness of battery health status prediction.

[0152] In some practical applications, the method provided in this application can be tested using general datasets provided by existing research. Based on cross-dataset validation results, the accuracy of the method in cross-cell SOH estimation is comparable to domain adaptation or fine-tuning methods that require additional training data. Furthermore, this application can also analyze the impact of the design of each component on its effectiveness based on ablation experiments.

[0153] For PGTPT, this application initializes the prompt to the default manually created prompt "Analyze Series Data" and optimizes the corresponding text tags in the text input embedding space. This application optimizes the prompt to minimize the PGTPT loss to one step using an AdamW optimizer with a learning rate of 0.0001. After obtaining the output features O~∈R^P×D, this application can transform O~ into a one-dimensional tensor of length P×D, input it into a fully connected layer, linearly project it downwards to SOH y^, and linearly project it upwards to the complete battery charging curve.

[0154] As mentioned above, in this application, the input charging curve is constrained by the physical constraints, and the component structure and component features of the input translation layer are obtained by training based on the constrained input charging curve and backpropagation features.

[0155] The propagation path of the backpropagation features includes: from the output mapping layer to the output feature layer of the model backbone to the main body of the model backbone to the input translation layer;

[0156] The equivalent circuit model is used to constrain the complete output charging curve of the feature reconstruction layer. Based on the constrained complete output charging curve, prompt words are trained and obtained. The prompt words and the sequence words are input into the model backbone of the basic language model to obtain the output features of the current model backbone. Based on the output features, the component structure and component features of the feature reconstruction layer and the output mapping layer are trained and obtained.

[0157] Example 2:

[0158] This application provides a battery health status prediction device based on a language model, the device comprising:

[0159] The determination module is used to determine the target battery data;

[0160] The target battery data includes the change of at least one electrical parameter of the target battery over time;

[0161] The processing module is used to process the target battery data to obtain a first input feature corresponding to the input form of the target language model;

[0162] The target language model is obtained by training a base language model through cue learning. The base language model includes a pre-trained model backbone, adjustable components, and a cue. The pre-trained model backbone includes a GPT-2 model backbone. The adjustable components include an input translation layer, an output mapping layer, and a feature reconstruction layer. The cue is used to generate cue words, which are used to indicate the second input features of the model backbone.

[0163] Furthermore, the input translation layer is used to obtain the corresponding text representation sequence features based on the input features corresponding to the battery training data; the feature reconstruction layer is used to obtain the complete output charging curve based on the output features of the target language model; the complete output charging curve is used to optimize the prompt words corresponding to the prompt learning process; and the output mapping layer is used to obtain the battery health status of the corresponding battery based on the output features of the target language model.

[0164] The processing module is further configured to input the first input feature and the second input feature into the target language model to obtain the battery health status corresponding to the target battery;

[0165] And, the processing module provides learning prompts in the following ways:

[0166] Determine the battery training data and input the battery training data into the base language model;

[0167] The battery training data includes the input charging curves for each battery.

[0168] Determine the physical constraints;

[0169] The physical constraints are used to indicate the constraints on the physical model and electrical parameters of the target battery.

[0170] Based on the aforementioned physical constraints, determine the optimization objective function;

[0171] Based on the optimization objective function, generate the output charging curve;

[0172] During the training phase, the component structure and component features of each of the adjustable components and the prompter are adjusted according to the output charging curve; and during the testing phase, the component structure and component features of the prompter are adjusted according to the output charging curve.

[0173] As an optional implementation, the method by which the determining module determines the specific physical constraints includes:

[0174] Based on the first-order RC equivalent circuit model, the first formula is determined;

[0175] The first formula is used to indicate the electrical behavior of the first-order RC equivalent circuit model, and the first formula includes:

[0176]

[0177] Where u is the terminal voltage, I is the equivalent current, and θ1(T) and θ2(T) are temperature-related functions used to indicate the electrical behavior of the first-order RC equivalent circuit model;

[0178] And, determining the optimization objective function based on the physical constraints includes:

[0179] Determine the unsupervised loss function based on the second formula;

[0180] The second formula includes:

[0181]

[0182] The objective function is used to indicate the second norm based on the physical constraints, formed by the output values ​​and corresponding input values ​​of each sequence on the output charging curve, and the optimization condition of minimizing the sum of the products of the first optimization coefficient λ and the second norm based on the first formula and the physical constraints.

[0183] As an optional implementation, the specific method by which the processing module generates the output charging curve based on the optimization objective function includes:

[0184] Based on the battery training data, determine the encoding sequence;

[0185] The preset data in the encoded sequence is masked to generate target encoded data;

[0186] Based on the optimization objective function, the target encoded data is expanded to generate an output charging curve.

[0187] As an optional implementation, the first input feature includes sequence words, the battery training data includes input charging curves, and the specific method by which the processing module determines the battery training data and inputs the battery training data into the basic language model includes:

[0188] Obtain the input charging curves from the target battery dataset, and determine the corresponding time series data based on each input charging curve;

[0189] Based on the time series data, an input vector is determined, and the corresponding text representation sequence features are determined by the input translation layer using a multi-head attention mechanism.

[0190] Based on the text representation sequence features, sequence words are determined, and the sequence words are input into the model backbone of the basic language model.

[0191] As an optional implementation, the processing module is further configured to:

[0192] The input charging curve is constrained by the physical constraints, and the component structure and component features of the input translation layer are obtained by training based on the constrained input charging curve and backpropagation features.

[0193] The propagation path of the backpropagation features includes: from the output mapping layer to the output feature layer of the model backbone to the main body of the model backbone to the input translation layer;

[0194] The equivalent circuit model is used to constrain the complete output charging curve of the feature reconstruction layer. Based on the constrained complete output charging curve, prompt words are trained and obtained. The prompt words and the sequence words are input into the model backbone of the basic language model to obtain the output features of the current model backbone. Based on the output features, the component structure and component features of the feature reconstruction layer and the output mapping layer are trained and obtained.

[0195] For details on the various implementation methods and technical effects on the device side, please refer to the relevant descriptions on the corresponding method side; they will not be repeated here.

[0196] Example 3:

[0197] The present invention also provides an electronic device, which includes a processor, a memory, a bus, and a computer program stored in the memory and executable on the processor. The processor includes one or more processing cores, and the memory is connected to the processor via the bus. The memory is used to store program instructions, and when the processor executes the computer program, it implements the steps in the above-described method embodiment of Embodiment 1 of the present invention.

[0198] Furthermore, as an executable solution, the electronic device can be a computer unit, which can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. The computer unit may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described structure of the computer unit is merely an example and does not constitute a limitation on the computer unit. It may include more or fewer components, or combine certain components, or use different components. For example, the computer unit may also include input / output devices, network access devices, buses, etc., and this embodiment of the invention does not limit this.

[0199] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the computer unit, connecting various parts of the entire computer unit via various interfaces and lines.

[0200] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer unit by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0201] Example 4:

[0202] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the embodiments of the present invention.

[0203] If the modules / units integrated in the computer unit are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0204] Example 5:

[0205] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps described above.

[0206] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. A method for predicting battery health status based on a language model, characterized in that, The method includes: Determine the target battery data; The target battery data includes the change of at least one electrical parameter of the target battery over time; The target battery data is processed to obtain a first input feature corresponding to the input form of the target language model; The target language model is obtained by training a base language model through cue learning. The base language model includes a pre-trained model backbone, adjustable components, and a cue. The pre-trained model backbone includes a GPT-2 model backbone. The adjustable components include an input translation layer, an output mapping layer, and a feature reconstruction layer. The cue is used to generate cue words, which are used to indicate the second input features of the model backbone. Furthermore, the input translation layer is used to obtain the corresponding text representation sequence features based on the input features corresponding to the battery training data; the feature reconstruction layer is used to obtain the complete output charging curve based on the output features of the target language model; the complete output charging curve is used to optimize the prompt words corresponding to the prompt learning process; and the output mapping layer is used to obtain the battery health status of the corresponding battery based on the output features of the target language model. The first input feature and the second input feature are input into the target language model to obtain the battery health status corresponding to the target battery; And, the cue learning includes: Determine the battery training data and input the battery training data into the base language model; The battery training data includes the input charging curves for each battery. Determine the physical constraints; The physical constraints are used to indicate the constraints on the physical model and electrical parameters of the target battery. Based on the aforementioned physical constraints, determine the optimization objective function; Based on the optimization objective function, generate the output charging curve; During the training phase, the component structure and component features of each of the adjustable components and the prompter are adjusted according to the output charging curve; and during the testing phase, the component structure and component features of the prompter are adjusted according to the output charging curve.

2. The method according to claim 1, characterized in that, The determination of physical constraints includes: Based on the first-order RC equivalent circuit model, the first formula is determined; The first formula is used to indicate the electrical behavior of the first-order RC equivalent circuit model, and the first formula includes: Where u is the terminal voltage, I is the equivalent current, θ1(T) and θ2(T) are temperature-related functions used to indicate the electrical behavior of the first-order RC equivalent circuit model, and T is the temperature; And, determining the optimization objective function based on the physical constraints includes: Determine the unsupervised loss function based on the second formula; The second formula includes: The objective function is used to indicate the second norm based on the physical constraints, formed by the output values ​​and corresponding input values ​​of each sequence on the output charging curve, and the optimization condition of minimizing the sum of the products of the first optimization coefficient λ and the second norm based on the first formula and the physical constraints; where X′ is the complete battery charging curve and Xtest is the test sample.

3. The method according to claim 2, characterized in that, The step of generating the output charging curve based on the optimized objective function includes: Based on the battery training data, determine the encoding sequence; The preset data in the encoded sequence is masked to generate target encoded data; Based on the optimization objective function, the target encoded data is expanded to generate an output charging curve.

4. The method according to claim 2, characterized in that, The first input feature includes sequence words, the battery training data includes input charging curves, and determining the battery training data and inputting the battery training data into the base language model includes: Obtain the input charging curves from the target battery dataset, and determine the corresponding time series data based on each input charging curve; Based on the time series data, an input vector is determined, and the corresponding text representation sequence features are determined by the input translation layer using a multi-head attention mechanism. Based on the text representation sequence features, sequence words are determined, and the sequence words are input into the model backbone of the basic language model.

5. The method according to claim 4, characterized in that, The method further includes: The input charging curve is constrained by the physical constraints, and the component structure and component features of the input translation layer are obtained by training based on the constrained input charging curve and backpropagation features. The propagation path of the backpropagation features includes: from the output mapping layer to the output feature layer of the model backbone to the main body of the model backbone to the input translation layer; The equivalent circuit model is used to constrain the complete output charging curve of the feature reconstruction layer. Based on the constrained complete output charging curve, prompt words are trained and obtained. The prompt words and the sequence words are input into the model backbone of the basic language model to obtain the output features of the current model backbone. Based on the output features, the component structure and component features of the feature reconstruction layer and the output mapping layer are trained and obtained.

6. A battery health status prediction device based on a language model, characterized in that, The device includes: The determination module is used to determine the target battery data; The target battery data includes the change of at least one electrical parameter of the target battery over time; The processing module is used to process the target battery data to obtain a first input feature corresponding to the input form of the target language model; The target language model is obtained by training a base language model through cue learning. The base language model includes a pre-trained model backbone, adjustable components, and a cue. The pre-trained model backbone includes a GPT-2 model backbone. The adjustable components include an input translation layer, an output mapping layer, and a feature reconstruction layer. The cue is used to generate cue words, which are used to indicate the second input features of the model backbone. Furthermore, the input translation layer is used to obtain the corresponding text representation sequence features based on the input features corresponding to the battery training data; the feature reconstruction layer is used to obtain the complete output charging curve based on the output features of the target language model; the complete output charging curve is used to optimize the prompt words corresponding to the prompt learning process; and the output mapping layer is used to obtain the battery health status of the corresponding battery based on the output features of the target language model. The processing module is further configured to input the first input feature and the second input feature into the target language model to obtain the battery health status corresponding to the target battery; And, the specific methods by which the processing module performs prompting learning include: Determine the battery training data and input the battery training data into the base language model; The battery training data includes the input charging curves for each battery. Determine the physical constraints; The physical constraints are used to indicate the constraints on the physical model and electrical parameters of the target battery. Based on the aforementioned physical constraints, determine the optimization objective function; Based on the optimization objective function, generate the output charging curve; During the training phase, the component structure and component features of each of the adjustable components and the prompter are adjusted according to the output charging curve; and during the testing phase, the component structure and component features of the prompter are adjusted according to the output charging curve.

7. The apparatus according to claim 6, characterized in that, The specific method by which the processing module generates the output charging curve based on the optimized objective function includes: Based on the battery training data, determine the encoding sequence; The preset data in the encoded sequence is masked to generate target encoded data; Based on the optimization objective function, the target encoded data is expanded to generate an output charging curve.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing at least one program, the at least one program being executed by the processor to implement the method as described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, which is executed by a processor to implement the method as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-5.

Images