Big language model battery evaluation method and device based on multi-modal fusion

Abstract

The invention discloses a large language model battery evaluation method and device based on multi-modal fusion, and relates to the technical field of battery intelligent management. The method comprises the steps that according to time sequence operation data, an improved multi-scale gating time sequence model is used for time sequence feature extraction, and operation time sequence features are obtained; according to the log text data and the monitoring image data, performing semantic feature extraction by using a multi-modal large language model to obtain fused semantic features; performing medium-term fusion reasoning by using an improved large language model according to the operation time sequence features and the fusion semantic features to obtain a battery evaluation result; generating a battery risk detection result and a corresponding explanatory text according to a preset alarm threshold value and a battery evaluation result based on the post-masking time sequence attention flow and the post-masking semantic attention flow; and carrying out risk visualization according to the battery risk detection result and the corresponding explanatory text. The invention relates to a high-reliability battery evaluation method based on a multi-modal large language model.

Description

Technical Field

[0001] This invention relates to the field of intelligent battery management technology, and in particular to a battery evaluation method and apparatus based on a large language model using multimodal fusion. Background Technology

[0002] Lithium-ion batteries are core energy storage units in key areas such as new energy vehicles, large-scale energy storage power stations, consumer electronics, and unmanned equipment. Their safety, reliability, and economy are of paramount importance. The battery's State of Health (SOH) and Remaining Useful Life (RUL) are two core indicators for evaluating its performance, developing maintenance strategies, and predicting failure risks. Therefore, achieving accurate and real-time prediction of SOH and RUL has become a research hotspot and technical challenge in the field of Battery Management Systems (BMS).

[0003] Existing methods for predicting battery SOH and RUL can be mainly divided into two categories: model-driven and data-driven. Model-driven methods rely on complex electrochemical models or equivalent circuit models. Although they have good physical interpretation, the modeling process is complex, parameter identification is difficult, and they are not adaptable to batteries with different chemical systems and operating conditions, resulting in poor versatility.

[0004] With the development of artificial intelligence technology, data-driven methods, especially those based on deep learning, have become mainstream. Researchers widely employ models such as Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Convolutional Neural Network (CNN) to predict future states of batteries by learning from a large amount of historical battery operating data, such as time series data on voltage, current, and temperature.

[0005] However, these methods still have the following significant limitations. Traditional deep learning models rely almost entirely on structured time-series electrical signal data. They cannot effectively utilize and understand other modal information that has a significant impact on battery health, such as unstructured maintenance logs (e.g., electrolyte replacement, excessive voltage difference in a cell, equipment specifications), and visual image data that can intuitively reflect the physical safety status, such as swelling on the battery surface, leakage, or local hot spots in infrared thermal imaging. This lack of information prevents predictive models from forming a comprehensive judgment on battery assessment, especially when predicting sudden failures caused by physical reasons.

[0006] Traditional time-series models excel at capturing patterns in data sequences, but lack the ability to understand complex causal relationships. For example, the model cannot logically correlate an overcharge event mentioned in the operation log with subsequent accelerated capacity decay, nor can it adjust the weighting of the battery's chemical system in assessing its impact on temperature.

[0007] Deep learning models are often considered black boxes; they can provide a predicted value but cannot offer the specific reasons for that prediction. In real-world operational scenarios, users not only want to know the State of Health (SOH) value but also why the SOH decreased and what factors led to the increased risk, such as increased internal resistance or a hotspot. This lack of interpretability significantly limits the credibility and application value of the predictions in practical decision-making.

[0008] Multimodal large language models (MLLMs), represented by generative pre-trained transformers (GPT) and Gemini models, have demonstrated powerful cross-modal understanding, context learning, and complex logical reasoning capabilities. They can simultaneously process and fuse text, images, and structured data, and generate logical analyses and explanations in the form of natural language.

[0009] Therefore, there is a lack of a novel prediction framework in the existing technology that combines the advantages of traditional time series models in processing long-sequence data with the powerful capabilities of multimodal large language models in information fusion, deep understanding and interpretable reasoning, so as to achieve a panoramic, high-precision and interpretable battery evaluation method for battery status. Summary of the Invention

[0010] To address the technical problems of existing battery health prediction methods, such as limited information dimensions, one-sided evaluation, lack of contextual understanding and logical reasoning ability, and poor interpretability, this invention provides a battery evaluation method and apparatus based on a large language model using multimodal fusion. The technical solution is as follows:

[0011] On the one hand, a method for evaluating the battery performance of a large language model based on multimodal fusion is provided. This method is implemented by a large language model battery evaluation device and includes:

[0012] Based on the battery testing platform, vehicle battery management system and remote sensor monitoring system, data of the battery to be evaluated is collected and preprocessed in different cycles to obtain the time-series operation data, log text data and monitoring image data of the battery to be evaluated.

[0013] Based on the time-series runtime data, an improved multi-scale gated time-series model is used to extract time-series features and obtain runtime time-series features.

[0014] Based on log text data and monitoring image data, a multimodal large language model is used to extract semantic features and obtain fused semantic features.

[0015] Based on runtime sequence features and fused semantic features, an improved large language model is used for mid-term fusion inference to obtain battery evaluation results, masked temporal attention flow, and masked semantic attention flow.

[0016] Based on post-masked temporal attention flow and post-masked semantic attention flow, battery risk detection results and corresponding explanatory text are generated according to preset alarm thresholds and battery evaluation results; risk visualization is performed based on battery risk detection results and corresponding explanatory text.

[0017] On the other hand, a battery evaluation device for large language models based on multimodal fusion is provided. This device is applied to a battery evaluation method for large language models based on multimodal fusion. The device includes:

[0018] The multimodal data acquisition module is used to collect and preprocess data of the battery to be evaluated in different cycles based on the battery testing platform, vehicle battery management system and remote sensor monitoring system, to obtain the time-series operation data, log text data and monitoring image data of the battery to be evaluated.

[0019] The time series feature extraction module is used to extract time series features from time series runtime data using an improved multi-scale gated time series model to obtain runtime time series features.

[0020] The semantic feature extraction module is used to extract semantic features from log text data and monitoring image data using a multimodal large language model to obtain fused semantic features.

[0021] The feature fusion reasoning module is used to perform mid-term fusion reasoning based on runtime sequence features and fused semantic features using an improved large language model to obtain battery evaluation results, masked temporal attention flow, and masked semantic attention flow.

[0022] The battery risk alarm module is used to generate battery risk detection results and corresponding explanatory text based on the masked temporal attention flow and the masked semantic attention flow, according to the preset alarm threshold and battery evaluation results; and to visualize the risk based on the battery risk detection results and the corresponding explanatory text.

[0023] On the other hand, a large language model battery evaluation device is provided, the large language model battery evaluation device comprising: a processor; a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, any one of the above-described large language model battery evaluation methods based on multimodal fusion is implemented.

[0024] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for evaluating large language model batteries based on multimodal fusion.

[0025] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0026] This invention achieves dual innovation in model structure and inference method by introducing a multimodal and large language model fusion mechanism into the traditional battery prediction process. Through an improved multi-scale gated temporal model, it captures dynamic features of battery degradation at different time scales, significantly improving the ability to identify sudden degradation and long-term aging. A bidirectional cross-attention mid-term fusion layer is added to the inference structure of the multimodal large language model, realizing bidirectional constraint and collaborative modeling of temporal features and textual / image semantic features, enabling the model to simultaneously possess physical law perception and semantic understanding capabilities. An adaptive gating weighting mechanism is introduced to balance the fusion depth of semantic and physical information, effectively preventing excessive information coupling. Finally, a result confidence analysis module enables the credibility assessment and visual early warning of prediction results. These improvements enable this invention to achieve higher accuracy, stronger robustness, and better interpretability in battery health status and remaining life prediction, breaking through the limitations of traditional single-modal or unidirectional fusion methods. This invention is a high-confidence battery evaluation method based on a multimodal large language model. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of a large language model battery evaluation method based on multimodal fusion provided by an embodiment of the present invention;

[0029] Figure 2 This is a block diagram of a large language model battery evaluation device based on multimodal fusion provided in an embodiment of the present invention;

[0030] Figure 3This is a schematic diagram of the structure of a large language model battery evaluation device provided in an embodiment of the present invention. Detailed Implementation

[0031] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0032] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0033] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0034] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0035] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0036] This invention provides a method for evaluating the battery performance of a large language model based on multimodal fusion. This method can be implemented by a large language model battery evaluation device, which can be a terminal or a server. Figure 1 The flowchart shown is for a large language model battery evaluation method based on multimodal fusion. The processing flow of this method may include the following steps:

[0037] S1. Based on the battery testing platform, vehicle battery management system and remote sensor monitoring system, collect and preprocess the battery data to be evaluated in different cycle periods to obtain the time-series operation data, log text data and monitoring image data of the battery to be evaluated.

[0038] In one feasible implementation, the present invention collects timing data of multiple batteries at different cycle periods, including but not limited to charging capacity. Discharge capacity Charging and discharging current ,Voltage ,energy ,temperature Cycle time Charge and discharge efficiency These parameters are collected and uploaded by the battery testing platform, vehicle battery management system, or remote sensing and monitoring system, supporting data input in single-cell or multi-cell scenarios.

[0039] Since the collected data may have issues such as inconsistent format, missing data, and noise, systematic preprocessing of the original data is necessary to improve the effectiveness and robustness of subsequent model processing.

[0040] Unit standardization: Convert all collected data to International Standard Units (SI units), such as current in amperes (A) and energy in watt-hours (Wh), to ensure comparability between different parameters.

[0041] Missing value imputation: For missing values ​​caused by device communication errors, sensor delays, etc., linear interpolation or moving average methods are used to fill in the missing values. Let the first... The missing value at time step Then it can be estimated using the following linear interpolation formula (1):

[0042] (1);

[0043] Time alignment: Since some parameters may be acquired at different sampling frequencies, such as temperature data which may be updated every minute, while voltage and current data may be updated every second, it is necessary to unify the resampling frequency. All sequences are synchronized to a unified time base using interpolation or downsampling methods.

[0044] Noise suppression: Abnormal fluctuations are processed using filtering algorithms, such as those applied to the original voltage signal. The sliding window averaging filter is applied, and the process is shown in equation (2) below:

[0045] (2);

[0046] Among them, window width It can be set according to the actual fluctuation frequency, such as .

[0047] Feature normalization: To avoid the interference of feature dimension differences on model weight calculation, all feature values ​​need to be scaled to a uniform range. The Z-score normalization method is usually adopted. The process is shown in the following equation (3):

[0048] (3);

[0049] in, The sample mean. The standard deviation is denoted as .

[0050] Unstructured text data related to the battery is obtained from databases, log files, or API interfaces, such as inspection records filled out by maintenance personnel, fault alarm logs automatically generated by the system, and battery specifications. First, word segmentation, symbol removal, and numerical unit standardization are performed, as shown in the following formula (4):

[0051] (4);

[0052] in, This represents the embedding vector of each word in the text, obtained through a word embedding function.

[0053] The word sequence is mapped to a vector representation as shown in equation (5):

[0054] (5);

[0055] in, This is a word embedding function, which can be an embedding interface provided by Word2Vec, BERT, or the Large Language Model API.

[0056] By deploying infrared thermal imagers and high-definition cameras on battery compartments or inspection robots, visual image data of batteries or battery packs can be captured at key time points (such as the end of charging or after full discharge).

[0057] For thermal imaging images, pixel normalization is first performed, as shown in equation (6):

[0058] (6);

[0059] in, These are the original pixel values. , These are the mean and standard deviation, respectively.

[0060] Extracting statistical features, such as average temperature and maximum temperature difference, is the process shown in equation (7) below:

[0061] (7);

[0062] S2. Based on the time-series runtime data, use the improved multi-scale gated time-series model to extract time-series features and obtain runtime time-series features;

[0063] Among them, the improved multi-scale gated temporal model is obtained by introducing multi-scale convolutional kernels and a gated weight sharing mechanism on the basis of gated recurrent units or long short-term memory networks.

[0064] The improved multi-scale gated temporal model includes a residual fusion layer, a multi-scale extraction layer, and a gate selection layer.

[0065] In one feasible implementation, the improved multi-scale gated temporal model introduces a multi-scale convolutional kernel and a gated weight sharing mechanism on the basis of traditional gated recurrent units or long short-term memory networks. This is used to capture short-term dynamics and long-term degradation trends at different time scales simultaneously, and to improve training stability and feature transfer capability through residual connection structures.

[0066] A composite loss function is introduced during the training process of the improved multi-scale gated time series model to ensure that the extracted time series feature summary better conforms to the physical laws of battery degradation. The composite loss function consists of a prediction error term, a monotonicity regularization term, and a physical law constraint term, as shown in equation (8):

[0067] (8);

[0068] in, For prediction error terms, such as mean squared error based on SOH or RUL; As a monotonic regularization, the SOH used to constrain prediction decreases monotonically with the number of cycles, and its mathematical expression is as follows (9):

[0069] (9);

[0070] in, Indicates the first Predicted SOH for the next cycle; As a constraint of physical laws, it is used to constrain the relationship between capacity decay and internal resistance increase, ensuring that the prediction results conform to the battery degradation mechanism.

[0071] By using the aforementioned composite loss function, prediction accuracy is optimized while maintaining physical interpretability and robustness to changes in battery health status.

[0072] Optionally, based on the time-series runtime data, an improved multi-scale gated time-series model is used to extract time-series features to obtain runtime time-series features, including:

[0073] Differential processing is performed on the time-series runtime data to obtain differential time-series data; the differential time-series data is concatenated with the time-series runtime data to obtain extended time-series runtime data;

[0074] Based on a preset multi-scale set, adaptive fusion coding is performed on extended time-series running data to obtain fused time-series features;

[0075] Based on the gated residual structure, residual refinement is performed on the fused temporal features to obtain runtime temporal features.

[0076] In one feasible implementation, long-series time-series data (such as battery charge-discharge capacity, energy efficiency, internal resistance, etc., spanning hundreds to thousands of cycles) are unsuitable for direct modeling and prediction due to significant redundancy and noise. Therefore, dynamic feature compression is required to preserve key degradation trend information and improve the processing efficiency of downstream models.

[0077] For each cycle k, calculate the multi-cycle ΔQ vector. (where p is the period span), these difference sequences are concatenated with the original time-series running data to form extended time-series running data. This significantly improves the perception of changes in degradation rate. Encoders at several scales (windows) are built in parallel to capture short-term fluctuations, medium-term patterns, and long-term trends, respectively, and then adaptively fused. Given the original time series... Set up the scale set. ,(For example That is, the original step size. For each scale, downsampling and encoding are performed as follows: (10)

[0078] (10);

[0079] in, It can be an LSTM / Transformer (heterogeneous). Then use scale-adaptive weights. Aggregation, the process is as follows (11):

[0080] (11);

[0081] in, A summary of the sequence encoded at the m-th scale (such as the last hidden state or average pooling). , These are learnable parameters. This mechanism allows the model to adaptively determine which layer is more important in the short / medium / long term.

[0082] Gated residual structures are used on multi-scale results to form runtime sequence features. The process is as follows: (12) and (13):

[0083] (12);

[0084] (13);

[0085] in, Preset metadata (such as battery model, temperature baseline). and For learnable transformations (linear or small multilayer perceptrons). Gating. Controlling the retention of the original abstract or introducing residual information can improve sensitivity to mutations (such as sudden internal resistance jumps).

[0086] S3. Based on log text data and monitoring image data, use a multimodal large language model to extract semantic features and obtain fused semantic features;

[0087] Optionally, based on log text data and monitoring image data, a multimodal large language model is used to extract semantic features to obtain fused semantic features, including:

[0088] Based on the log text data, a text encoder is used to extract semantic features to obtain text semantic features.

[0089] Based on the monitored image data, a visual encoder is used to extract semantic features to obtain the image semantic features;

[0090] Based on the attention mechanism, feature fusion is performed on text semantic features and image semantic features to obtain fused semantic features.

[0091] In one feasible implementation, a pre-trained multimodal large language model is used in this step. Its core objective is to map heterogeneous image and text data into a unified semantic representation space, laying the foundation for subsequent intermediate fusion. This process fully leverages the cross-modal understanding capabilities of the multimodal large language model, acquired through pre-training on massive image-text pairs.

[0092] The preprocessed text data is input into the model's text encoder. This encoder, through its deep Transformer architecture and self-attention mechanism, performs context-aware deep semantic encoding on the input text, outputting a vector representation H_text that encapsulates the full text's semantics. This is fundamentally different from simple bag-of-words models or shallow word embeddings.

[0093] The preprocessed image data is input into the model's visual encoder (e.g., a Vision Transformer-based architecture). This visual encoder is co-optimized with the text encoder during the pre-training phase, enabling it to encode image content into feature vectors H_img that are semantically compatible with the text. For example, it can align "hot spots" in thermal imaging images with textual concepts such as "overheating" and "risk of thermal runaway" in semantic space.

[0094] Image modal features can also be extracted using convolutional neural networks, such as Residual Network (ResNet), Efficient Network (EfficientNet), or Convolutional Network Next (ConvNeXt) structures, to capture local spatial features in battery appearance images or thermal imaging data.

[0095] After dimensionality reduction and standardization, the image features extracted by the convolutional neural network are aligned with the text semantic features output by the language model in the feature space and jointly input into the subsequent fusion stage.

[0096] S4. Based on runtime sequence features and fused semantic features, use an improved large language model to perform mid-term fusion inference to obtain battery evaluation results, masked temporal attention flow, and masked semantic attention flow.

[0097] Optionally, based on runtime sequence features and fused semantic features, an improved large language model is used for mid-term fusion inference to obtain battery evaluation results, post-masked temporal attention streams, and post-masked semantic attention streams, including:

[0098] Based on the physical consistency attention constraint module, attention mask constraint calculation is performed according to runtime sequence features and fused semantic features to obtain the masked temporal attention flow and the masked semantic attention flow.

[0099] Based on the degradation stage discrimination module, the preset gate signal is dynamically adjusted according to the timing characteristics to obtain the optimized gate signal;

[0100] Based on the optimized gating signal, the runtime sequence features and the fused semantic features are fused according to the masked temporal attention flow and the masked semantic attention flow to obtain multimodal fused features;

[0101] Battery evaluation results are obtained by mapping based on multimodal fusion features.

[0102] In one feasible implementation, after obtaining runtime sequence features and fused semantic features, the data is aggregated into a unified data structure according to the input specifications of the target large language model API (e.g., JSON object, dictionary structure, or multimodal tensor format), so that the data of each modality are logically stored in parallel, maintaining their original independence.

[0103] The feature data of different modalities have not yet undergone semantic-level interaction or weighting processing. Instead, they are uniformly encapsulated in a structured and standardized manner to ensure that all evidence can be transmitted completely and without loss to the subsequent large language model API for feature alignment and fusion reasoning.

[0104] This invention adds a bidirectional cross-attention mid-stage fusion layer to the inference stage of a multimodal large language model, enabling dynamic interaction between battery degradation features extracted by traditional time-series models and semantic features within the large language model during inference. Compared to existing methods that only perform feature concatenation at the input end or decision-level fusion at the output end, this invention moves the modal interaction position forward to the mid-inference layer, allowing semantic generation and state inference processes to reference physical degradation information in real time, thereby achieving continuous constraint and enhancement of the inference process.

[0105] Feature association modeling is achieved through a bidirectional cross-attention mechanism, enabling the semantic modeling process to perceive the degradation trend of the battery with cycling. Simultaneously, temporal degradation modeling can obtain global explanatory information based on the semantic context, ultimately achieving synchronous modeling and consistent expression of semantic information and physical laws. The attention results from these two directions are concatenated and linearly transformed to obtain fused features. Then, the data is input into the subsequent inference layer of the language model to complete the prediction output.

[0106] The results of the two attention streams are fused by gating weighting to balance the importance of semantic and physical features, as shown in Equation (14):

[0107] (14);

[0108] in: Use the Sigmoid activation function; This is the gate weight matrix; This indicates element-weighted; This indicates feature splicing.

[0109] This gating mechanism adaptively adjusts the fusion ratio of the two modalities based on the current task context, while preserving original modal features through residual connections to avoid information loss due to over-fusion. Final fused representation The input large language model is used for subsequent semantic reasoning to generate predictions and explanations for battery state of health (SOH) and remaining lifetime (RUL).

[0110] The feature representation output by the fusion inference module is mapped to the final SOH (scalar between 0 and 1) and RUL (integer value in units of iteration number) point prediction results through a fully connected layer or regression head.

[0111] Unlike existing multimodal fusion methods that only perform static modal splicing or unidirectional attention guidance at the input end, this invention moves the fusion position forward to the middle of the inference stage and constructs a bidirectional cross-attention structure of temporal → semantic and semantic → temporal, so that semantic information can explicitly supplement the battery degradation trend, and the degradation law can inversely constrain the semantic attention area, thereby achieving bidirectional consistency modeling between modalities.

[0112] The adaptive fusion mechanism based on degradation stage discrimination enables the fusion weights to be dynamically adjusted according to the degree of battery degradation, which solves the problem that the prediction error of existing methods increases significantly in the rapid decay stage and improves the model's stable prediction ability in the later stages of life.

[0113] The introduced physical consistency attention mask effectively suppresses attention connections that do not conform to the monotonic decay of capacity and the increase of internal resistance, avoiding invalid inference phenomena such as "capacity rebound" and "abnormal decrease of internal resistance", making the model inference process consistent with the electrochemical mechanism, and enhancing the interpretability and credibility of the model output.

[0114] Therefore, the bidirectional cross-attention mid-term fusion structure proposed in this invention not only improves the model's ability to characterize temporal decay information, but also enhances the physical constraints of semantic generation and decision-making processes. It achieves higher prediction accuracy, stability and interpretability in health status and remaining lifespan prediction tasks, which is significantly better than traditional one-way fusion and static weighting methods.

[0115] Optionally, based on the physical consistency attention constraint module, attention mask constraints are calculated according to runtime sequence features and fused semantic features to obtain the masked temporal attention flow and the masked semantic attention flow, including:

[0116] Based on the battery degradation mechanism, a physical consistency mask is generated according to the runtime sequence characteristics and fused semantic features to obtain the timing mask and semantic mask;

[0117] Based on runtime sequence features, temporal enhanced attention is calculated on the fused semantic features to obtain a temporal-driven attention stream; based on the temporal mask, the temporal-driven attention stream is masked to obtain a masked temporal attention stream.

[0118] Based on the fused semantic features, semantic enhancement attention is calculated on the runtime sequence features to obtain the semantic feedback attention stream; based on the semantic mask, the semantic feedback attention stream is masked to obtain the masked semantic attention stream.

[0119] In one feasible implementation, to avoid the model generating an attention distribution that does not conform to the battery degradation mechanism during the fusion inference process, this invention introduces a physically consistent attention mask matrix M in the bidirectional cross-attention calculation.

[0120] The mask matrix is ​​used to constrain the attention weights, ensuring that high-weighted connections are established only between feature pairs that satisfy the laws of monotonically decreasing capacity and monotonically increasing internal resistance. If a feature pair does not satisfy these physical laws, its corresponding attention weight is set to 0 or significantly reduced.

[0121] Let the initial attention weight matrix be A, then the attention weight A' after introducing the mask is as follows (15):

[0122] (15);

[0123] Here, ⊙ represents element-wise multiplication; when M(i,j)=0, it means that the characteristic relationship does not conform to the physical law of battery degradation, then A'(i,j) will approach 0 after softmax normalization and no longer affect the fusion output.

[0124] By introducing a physical consistency mask matrix, this invention can effectively avoid unreasonable inference errors such as "capacity recovery" and "internal resistance decline" generated by the model, and improve the consistency and interpretability of the model output results with the actual degradation mechanism.

[0125] The feature fusion step includes a semantic enhancement path dominated by temporal features and a physical feedback path dominated by semantic features.

[0126] Semantic enhancement path to improve the features output of multi-scale gated temporal models As a query, the semantic features output by a multimodal large language model As the Key and Value, they are used to enhance the modeling of temporal features through semantic features. The calculation formula is as follows (16):

[0127] (16);

[0128] in: is the learnable parameter matrix; d is the feature dimension; softmax is used to normalize the attention weights.

[0129] This mechanism enables the language model to dynamically focus on semantic feature regions related to battery degradation trends when generating temporal representations, thereby forming semantically enhanced temporal representations.

[0130] Physical feedback path with semantic features For queries, based on time-series features For the Key and Value, calculate the reverse attention to implement the physical constraint of temporal information on the semantic attention distribution. Its calculation formula is as follows (17):

[0131] (17);

[0132] in, These are independent parameters for the semantic feedback stream.

[0133] This structure enables the model to automatically learn the time-dependent patterns of battery degradation during the semantic modeling process, achieving physical consistency in semantic generation.

[0134] Optionally, based on the degradation stage discrimination module, the preset gating signal is dynamically adjusted according to the timing characteristics to obtain an optimized gating signal, including:

[0135] Based on the capacity characteristics and internal resistance characteristics of the runtime sequence characteristics, the capacity trend change characteristics and internal resistance trend change characteristics are calculated.

[0136] The current degradation stage is determined based on the characteristics of capacity trend changes and internal resistance trend changes.

[0137] Based on the current degradation stage, the preset gating signal is dynamically adjusted to obtain an optimized gating signal.

[0138] In one feasible implementation, in order to make the fusion strategy more closely match the actual aging process of the battery, the present invention adds a degradation stage discrimination module on top of the bidirectional attention mechanism to achieve dynamic adaptive adjustment of the fusion weight.

[0139] This module calculates the capacity and internal resistance variation characteristics obtained from a multi-scale gated timing model. .

[0140] according to and The changing trends divide the degradation process into a stable decay stage, a rapid decay stage, and a critical degradation stage. The adaptive adjustment of the fusion weights is reflected in the weighted fusion of the outputs of the two attention streams. Let the output of the time-driven stream be... The output of the semantic feedback stream is Then the final fusion features The gating is determined by the following gating formula (18):

[0141] (18);

[0142] Among them, gating signals It is a scalar weight (between 0 and 1) determined by the degradation stage.

[0143] During the stable decay phase: the semantic context may provide more forward-looking information, thus reducing... ,promote The weight.

[0144] During the rapid decay and critical degradation stages: abrupt changes in the physical signal become dominant, thus significantly increasing... Make the model more dependent on time-driven flow The captured strong degradation signal.

[0145] This mechanism ensures that the most effective fusion strategy can be adopted throughout the model's entire lifecycle.

[0146] Based on historical full-lifecycle data of batteries, dynamic thresholds for dividing stages are determined through statistical analysis. The first threshold is defined as the moving average of the capacity decay rate ΔQ and the internal resistance growth rate ΔR in historical data, plus one standard deviation. When the absolute value of ΔQ or ΔR consistently exceeds this threshold, the battery is determined to have entered the rapid degradation stage. The second threshold is defined as the moving average of ΔQ and ΔR in historical data plus two standard deviations, or reaching an absolute safety boundary preset according to the battery chemistry system (e.g., SOH < 0.7). When the absolute value of ΔQ or ΔR exceeds this threshold, the battery is determined to have entered the critical degradation stage. This dynamic threshold definition method based on data distribution statistics can adapt to different types of batteries under different operating conditions, without relying on manually set fixed values, thus improving the model's versatility and robustness.

[0147] S5. Based on the post-masked temporal attention flow and post-masked semantic attention flow, generate battery risk detection results and corresponding explanatory text according to the preset alarm threshold and battery evaluation results; perform risk visualization based on the battery risk detection results and corresponding explanatory text.

[0148] In one feasible implementation, based on the final battery evaluation results, the prediction uncertainty is quantified using at least one of the following methods:

[0149] Based on statistical variance, Dropout is enabled during model inference to perform multiple random forward propagations, and the variance of the prediction results is calculated as a confidence index. Based on historical residuals, a confidence interval is assigned to the current prediction value according to the historical prediction error distribution of the model on the validation set (such as a 95% confidence interval). The wider the interval, the lower the confidence level.

[0150] Based on the prediction results and confidence levels, execute at least one of the following anomaly detection rules:

[0151] Physical Range Out-of-Limit Detection: The effective range of SOH is set to [0.3, 1.05], and RUL is [0, maximum possible number of cycles]. If the predicted value exceeds this range, a "Physical Anomaly" alarm is triggered.

[0152] Confidence threshold detection: Set a confidence threshold θ_conf (e.g., variance threshold). When the variance of the prediction result is higher than θ_conf, the prediction is deemed "unreliable" and a "high uncertainty" alarm is triggered.

[0153] Abrupt change detection: Calculate the absolute value of the difference between the current predicted value and the predicted value of the previous period, |Δŷ|. If |Δŷ| exceeds a dynamic threshold set based on the historical rate of change, a "state abrupt change" alarm is triggered, indicating that a sudden degradation may occur.

[0154] Explanatory text is generated by extracting the attention weights in the bidirectional cross-attention mid-term fusion layer.

[0155] Based on the correspondence between the semantic features of the text and the original text tokens, the top K text segments with the highest attention weights are selected as the semantic basis for decision-making.

[0156] Based on the spatial mapping relationship between image features and original image pixels, a visual heatmap is generated on the original image from image feature regions with high attention weights, which serves as the visual basis for decision-making.

[0157] The descriptions of the aforementioned textual and visual evidence are combined into the final explanatory text output using predefined natural language templates or lightweight text generators.

[0158] The battery risk detection results are transformed into an intuitive visual interface and dynamic risk warnings are implemented. The system front-end interface (such as a web-based dashboard) should include at least the following linked components:

[0159] The SOH / RUL historical trend chart uses a line chart library (such as ECharts, D3.js) to plot the predicted SOH and RUL values ​​of the battery from the initial stage to the present. The chart should clearly label the health status threshold lines (e.g., SOH=0.8 as the lower limit of "good", SOH=0.7 as the "warning" threshold); confidence interval visualization uses semi-transparent colored bands to cover the confidence intervals on the trend chart, making the prediction uncertainty immediately apparent; a key evidence panel, linked to the trend chart, dynamically displays the Top-K textual evidence (high attention-weighted log entries) and the battery image covered by a heatmap (high attention-weighted image area) relied upon by the model's decision when the user clicks or hovers over a specific time point; a health status indicator displays the current health level (e.g., "normal", "sub-healthy", "warning") in real time, in the form of a dashboard or color labels (green / yellow / red).

[0160] The system backend continuously monitors the prediction results and detection indicators, and the early warning mechanism is as follows:

[0161] Triggering conditions: An alert will be triggered immediately when any of the following conditions are met:

[0162] SOH predicted value ≤ SOH warning threshold; RUL predicted value ≤ RUL maintenance cycle; any anomaly detection rule is triggered; confidence level ≤ confidence threshold;

[0163] Warning presentation: On the front-end interface, the corresponding trend chart area flashes red, the health status indicator turns red, and a warning dialog box pops up.

[0164] Background push notifications, through integration with message push services (such as email, SMS, enterprise internal communication tool APIs), automatically send warning information ("Battery ID: XXX, predicted SOH: 0.68, already below the threshold of 0.70, confidence level: 85%) to preset management personnel.

[0165] This invention achieves dual innovation in model structure and inference method by introducing a multimodal and large language model fusion mechanism into the traditional battery prediction process. Through an improved multi-scale gated temporal model, it captures dynamic features of battery degradation at different time scales, significantly improving the ability to identify sudden degradation and long-term aging. A bidirectional cross-attention mid-term fusion layer is added to the inference structure of the multimodal large language model, realizing bidirectional constraint and collaborative modeling of temporal features and textual / image semantic features, enabling the model to simultaneously possess physical law perception and semantic understanding capabilities. An adaptive gating weighting mechanism is introduced to balance the fusion depth of semantic and physical information, effectively preventing excessive information coupling. Finally, a result confidence analysis module enables the credibility assessment and visual early warning of prediction results. These improvements enable this invention to achieve higher accuracy, stronger robustness, and better interpretability in battery health status and remaining life prediction, breaking through the limitations of traditional single-modal or unidirectional fusion methods. This invention is a high-confidence battery evaluation method based on a multimodal large language model.

[0166] Figure 2 This is a block diagram of a large language model battery evaluation device based on multimodal fusion provided in an embodiment of the present invention. This device is used for a large language model battery evaluation method based on multimodal fusion. (Refer to...) Figure 2 The device includes a first acquisition module 310, a second acquisition module 320, an inspection module 330, and a training module 340. Wherein:

[0167] The multimodal data acquisition module 210 is used to collect and preprocess data of the battery to be evaluated in different cycles based on the battery testing platform, vehicle battery management system and remote sensor monitoring system, to obtain the time-series operation data, log text data and monitoring image data of the battery to be evaluated.

[0168] The time series feature extraction module 220 is used to extract time series features based on the time series running data using an improved multi-scale gated time series model to obtain runtime time series features;

[0169] The semantic feature extraction module 230 is used to extract semantic features based on log text data and monitoring image data using a multimodal large language model to obtain fused semantic features;

[0170] The feature fusion reasoning module 240 is used to perform mid-term fusion reasoning based on runtime sequence features and fused semantic features using an improved large language model to obtain battery evaluation results, masked temporal attention flow and masked semantic attention flow;

[0171] The battery risk alarm module 250 is used to generate battery risk detection results and corresponding explanatory text based on the masked temporal attention flow and the masked semantic attention flow, according to the preset alarm threshold and battery evaluation results; and to visualize the risk based on the battery risk detection results and the corresponding explanatory text.

[0172] Among them, the improved multi-scale gated temporal model is obtained by introducing multi-scale convolutional kernels and a gated weight sharing mechanism on the basis of gated recurrent units or long short-term memory networks.

[0173] The improved multi-scale gated temporal model includes a residual fusion layer, a multi-scale extraction layer, and a gate selection layer.

[0174] Optionally, the temporal feature extraction module 220 is further used for:

[0175] Differential processing is performed on the time-series runtime data to obtain differential time-series data; the differential time-series data is concatenated with the time-series runtime data to obtain extended time-series runtime data;

[0176] Based on a preset multi-scale set, adaptive fusion coding is performed on extended time-series running data to obtain fused time-series features;

[0177] Based on the gated residual structure, residual refinement is performed on the fused temporal features to obtain runtime temporal features.

[0178] Optionally, the semantic feature extraction module 230 is further used for:

[0179] Based on the log text data, a text encoder is used to extract semantic features to obtain text semantic features.

[0180] Based on the monitored image data, a visual encoder is used to extract semantic features to obtain the image semantic features;

[0181] Based on the attention mechanism, feature fusion is performed on text semantic features and image semantic features to obtain fused semantic features.

[0182] Optionally, the feature fusion inference module 240 is further used for:

[0183] Based on the physical consistency attention constraint module, attention mask constraint calculation is performed according to runtime sequence features and fused semantic features to obtain the masked temporal attention flow and the masked semantic attention flow.

[0184] Based on the degradation stage discrimination module, the preset gate signal is dynamically adjusted according to the timing characteristics to obtain the optimized gate signal;

[0185] Based on the optimized gating signal, the runtime sequence features and the fused semantic features are fused according to the masked temporal attention flow and the masked semantic attention flow to obtain multimodal fused features;

[0186] Battery evaluation results are obtained by mapping based on multimodal fusion features.

[0187] Optionally, the feature fusion inference module 240 is further used for:

[0188] Based on the battery degradation mechanism, a physical consistency mask is generated according to the runtime sequence characteristics and fused semantic features to obtain the timing mask and semantic mask;

[0189] Based on runtime sequence features, temporal enhanced attention is calculated on the fused semantic features to obtain a temporal-driven attention stream; based on the temporal mask, the temporal-driven attention stream is masked to obtain a masked temporal attention stream.

[0190] Based on the fused semantic features, semantic enhancement attention is calculated on the runtime sequence features to obtain the semantic feedback attention stream; based on the semantic mask, the semantic feedback attention stream is masked to obtain the masked semantic attention stream.

[0191] Optionally, the feature fusion inference module 240 is further used for:

[0192] Based on the capacity characteristics and internal resistance characteristics of the runtime sequence characteristics, the capacity trend change characteristics and internal resistance trend change characteristics are calculated.

[0193] The current degradation stage is determined based on the characteristics of capacity trend changes and internal resistance trend changes.

[0194] Based on the current degradation stage, the preset gating signal is dynamically adjusted to obtain an optimized gating signal.

[0195] This invention achieves dual innovation in model structure and inference method by introducing a multimodal and large language model fusion mechanism into the traditional battery prediction process. Through an improved multi-scale gated temporal model, it captures dynamic features of battery degradation at different time scales, significantly improving the ability to identify sudden degradation and long-term aging. A bidirectional cross-attention mid-term fusion layer is added to the inference structure of the multimodal large language model, realizing bidirectional constraint and collaborative modeling of temporal features and textual / image semantic features, enabling the model to simultaneously possess physical law perception and semantic understanding capabilities. An adaptive gating weighting mechanism is introduced to balance the fusion depth of semantic and physical information, effectively preventing excessive information coupling. Finally, a result confidence analysis module enables the credibility assessment and visual early warning of prediction results. These improvements enable this invention to achieve higher accuracy, stronger robustness, and better interpretability in battery health status and remaining life prediction, breaking through the limitations of traditional single-modal or unidirectional fusion methods. This invention is a high-confidence battery evaluation method based on a multimodal large language model.

[0196] Figure 3 This is a schematic diagram of the structure of a large language model battery evaluation device provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the large language model battery evaluation device may include the above-mentioned Figure 2 The large language model battery evaluation device shown is based on multimodal fusion. Optionally, the large language model battery evaluation device 310 may include a first processor 2001.

[0197] Optionally, the large language model battery evaluation device 310 may also include a memory 2002 and a transceiver 2003.

[0198] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0199] The following is combined Figure 3 A detailed introduction to each component of the large language model battery evaluation device 310 is provided below:

[0200] The first processor 2001 is the control center of the large language model battery evaluation device 310. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0201] Optionally, the first processor 2001 can perform various functions of the large language model battery evaluation device 310 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0202] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0203] In a specific implementation, as one example, the large language model battery evaluation device 310 may also include multiple processors, for example... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0204] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0205] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be accessed through the interface circuit of the large language model battery evaluation device 310. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0206] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0207] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0208] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected to the interface circuit of the large language model battery evaluation device 310. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0209] It should be noted that, Figure 3 The structure of the large language model battery evaluation device 310 shown in the figure does not constitute a limitation on the router. The actual large language model battery evaluation device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0210] Furthermore, the technical effect of the large language model battery evaluation device 310 can be referred to the technical effect of the large language model battery evaluation method based on multimodal fusion described in the above method embodiments, and will not be repeated here.

[0211] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor, etc.

[0212] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0213] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0214] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0215] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0216] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0217] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0218] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0219] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0220] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0221] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0222] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0223] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A large language model battery evaluation method based on multi-modal fusion, characterized by, The method comprises: Based on the battery test platform, the vehicle battery management system and the remote sensor monitoring system, the data of the battery to be evaluated is collected and preprocessed at different cycle periods to obtain the time sequence operation data, log text data and monitoring image data of the battery to be evaluated; According to the time sequence operation data, the improved multi-scale gated time sequence model is used for time sequence feature extraction to obtain the operation time sequence feature; According to the log text data and the monitoring image data, a multi-modal large language model is used for semantic feature extraction to obtain a fused semantic feature; According to the operation time sequence feature and the fused semantic feature, an improved large language model is used for medium-term fusion reasoning to obtain a battery evaluation result, a masked time sequence attention flow and a masked semantic attention flow; Based on the masked time sequence attention flow and the masked semantic attention flow, a battery risk detection result and corresponding explanatory text are generated according to a preset alarm threshold and the battery evaluation result; and risk visualization is performed according to the battery risk detection result and the corresponding explanatory text.

2. The multi-modal fusion based large language model battery evaluation method according to claim 1, characterized in that, The improved multi-scale gated time sequence model is obtained by introducing a multi-scale convolution kernel and a gated weight sharing mechanism based on a gated recurrent unit or a long short-term memory network. The improved multi-scale gated time sequence model comprises a residual fusion layer, a multi-scale extraction layer and a gated selection layer.

3. The multi-modal fusion based large language model battery evaluation method according to claim 1, characterized in that, The operation time sequence feature is obtained by using the improved multi-scale gated time sequence model to extract the time sequence feature from the time sequence operation data, comprising: The time sequence operation data is subjected to difference processing to obtain difference time sequence data; the difference time sequence data is spliced with the time sequence operation data to obtain extended time sequence operation data; Based on a preset multi-scale set, the extended time sequence operation data is subjected to adaptive fusion coding to obtain a fused time sequence feature; The fused time sequence feature is subjected to residual refinement processing based on a gated residual structure to obtain the operation time sequence feature.

4. The multi-modal fusion based large language model battery evaluation method according to claim 1, characterized in that, The fused semantic feature is obtained by using a multi-modal large language model to extract the semantic feature from the log text data and the monitoring image data, comprising: The text semantic feature is obtained by using a text encoder to extract the semantic feature from the log text data; The image semantic feature is obtained by using a visual encoder to extract the semantic feature from the monitoring image data; Based on an attention mechanism, the text semantic feature and the image semantic feature are fused to obtain the fused semantic feature.

5. The multi-modal fusion based large language model battery evaluation method according to claim 1, characterized in that, The battery evaluation result, the masked time sequence attention flow and the masked semantic attention flow are obtained by using the improved large language model to perform medium-term fusion reasoning on the operation time sequence feature and the fused semantic feature, comprising: Based on a physical consistency attention constraint module, the attention mask constraint calculation is performed on the operation time sequence feature and the fused semantic feature to obtain the masked time sequence attention flow and the masked semantic attention flow; Based on a degradation stage discrimination module, the preset gating signal is dynamically adjusted according to the time sequence feature to obtain an optimized gating signal; Based on the optimized gating signal, the operation time sequence feature and the fused semantic feature are fused according to the masked time sequence attention flow and the masked semantic attention flow to obtain a multi-modal fusion feature; According to the mapping of the multi-modal fusion features, a battery evaluation result is obtained.

6. The multi-modal fusion based large language model battery evaluation method according to claim 5, characterized in that, The physical consistency attention constraint module performs attention mask constraint calculation on the running time sequence features and the fused semantic features to obtain a masked time sequence attention flow and a masked semantic attention flow, including: Based on the battery degradation mechanism, the physical consistency mask is generated according to the running time sequence features and the fused semantic features to obtain a time sequence mask and a semantic mask; According to the running time sequence features, the fused semantic features are calculated by time sequence enhanced attention to obtain a time sequence driven attention flow; and according to the time sequence mask, the time sequence driven attention flow is processed by mask to obtain a masked time sequence attention flow. According to the fused semantic features, the running time sequence features are calculated by semantic enhanced attention to obtain a semantic feedback attention flow; and according to the semantic mask, the semantic feedback attention flow is processed by mask to obtain a masked semantic attention flow.

7. The multi-modal fusion based large language model battery evaluation method according to claim 5, characterized in that, The degradation stage discrimination module adjusts the preset gating signal according to the time sequence features to obtain an optimized gating signal, including: The capacity trend change feature and the internal resistance trend change feature are calculated according to the capacity feature and the internal resistance feature of the running time sequence features; The current degradation stage is determined according to the capacity trend change feature and the internal resistance trend change feature; The preset gating signal is dynamically adjusted based on the current degradation stage to obtain an optimized gating signal.

8. A large language model battery evaluation device based on multi-modal fusion, the large language model battery evaluation device based on multi-modal fusion is used to realize the large language model battery evaluation method based on multi-modal fusion as any one of claims 1-7, characterized in that, The device comprises: A multi-modal data acquisition module is configured to acquire and preprocess battery data of a battery to be evaluated based on a battery test platform, a vehicle battery management system, and a remote sensor monitoring system at different cycle periods to obtain time sequence running data, log text data, and monitoring image data of the battery to be evaluated. A time sequence feature extraction module is configured to extract time sequence features from the time sequence running data using an improved multi-scale gated time sequence model to obtain running time sequence features. A semantic feature extraction module is configured to extract semantic features from the log text data and the monitoring image data using a multi-modal large language model to obtain fused semantic features. A feature fusion inference module is configured to perform mid-term fusion inference on the running time sequence features and the fused semantic features using an improved large language model to obtain a battery evaluation result, a masked time sequence attention flow, and a masked semantic attention flow. A battery risk alarm module is configured to generate a battery risk detection result and corresponding explanatory text based on the masked time sequence attention flow and the masked semantic attention flow according to a preset alarm threshold and the battery evaluation result, and to visualize the risk based on the battery risk detection result and the corresponding explanatory text.

9. A large language model battery evaluation device, characterized by, The large language model battery evaluation device comprises: A processor; A memory having computer readable instructions stored thereon, which, when executed by the processor, implement the method of any one of claims 1 to 7.

10. A computer readable storage medium, characterized in that, The computer readable storage medium stores program code that can be invoked and executed by the processor to implement the method of any one of claims 1 to 7.

Images