A battery storage monitoring method and system based on deep learning

Abstract

The application provides a battery storage monitoring method and system based on deep learning, wherein multi-modal monitoring data are acquired, and thermal imaging features, visible light features and time sequence features are respectively extracted through a ResNet50 model, a MobileNetV3 model and a 1D-CNN model, and weighted fusion is performed through cross-modal attention, so that dynamic weighted fusion of image features and time sequence features is realized, the correlation between different types of data is fully mined, and the accuracy of identification and prediction is improved; and through a Transformer time sequence prediction model, thermal runaway precursor identification and time sequence trend prediction are realized, long-term time sequence dependency of multi-modal data is learned, abnormal change trends and risk probabilities in the next 5-30 minutes can be inferred in advance, early warning time reaches 15-25 minutes, disposal time left for workers is greatly increased, and risk expansion is effectively avoided.

Description

Technical Field

[0001] This invention relates to the fields of battery storage safety monitoring and artificial intelligence technology, and in particular to a battery storage monitoring method and system based on deep learning. Background Technology

[0002] With the rapid development of the new energy industry, the production and storage scale of various batteries, such as lithium batteries and storage batteries, is constantly expanding, making safety management in battery storage a top priority. During storage, batteries are highly susceptible to thermal runaway due to various factors, including aging, fluctuations in ambient temperature and humidity, external impacts, packaging damage, short circuits, and residual overcharging. Early signs of thermal runaway include initial bulging, localized abnormal temperature rises, trace amounts of smoke, minor electrolyte leakage, and abnormally high VOC (volatile organic compound) concentrations. If these signs are not identified and addressed promptly, they can escalate into fires and explosions, causing serious casualties, property damage, and environmental hazards. Currently, existing battery storage monitoring technologies have the following core deficiencies: Most monitoring systems rely on a single type of sensor (such as a temperature and humidity sensor or a single camera) or a single camera, which cannot comprehensively collect multi-dimensional features of the battery status. This results in a weak ability to identify subtle signs of thermal runaway, such as initial bulging and minor leakage, and a high risk of missed detections.

[0003] Existing technologies mostly use fixed thresholds to trigger early warnings, which can only issue alarms when abnormal phenomena are already quite obvious (such as a sharp rise in temperature or a large amount of smoke). They cannot identify early warning signs in advance, let alone predict the future trend of abnormal changes. The early warning time is insufficient, leaving limited time for staff to deal with the situation.

[0004] Existing monitoring systems mostly process the collected image and sensor data independently, failing to achieve deep fusion of multimodal data and fully explore the correlation between different types of data. This results in low accuracy and high false alarm rate in anomaly identification, making it difficult to adapt to complex warehousing environments.

[0005] Therefore, there is an urgent need for a battery storage monitoring solution that can accurately identify precursors to thermal runaway, predict abnormal trends in advance, and provide tiered early warnings. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a battery storage monitoring method and system based on deep learning. This invention uses a deep learning model to accurately identify precursors to thermal runaway and predict abnormal trends in advance, while also enabling tiered early warning systems, thereby further reducing the safety risks of battery storage.

[0007] In a first aspect, the present invention provides a battery storage monitoring method based on deep learning, comprising the following steps: S1) Acquire a multimodal monitoring data stream including thermal imaging image sequence of the battery storage area, visible light video frame sequence, ambient temperature and humidity data, and ambient VOC gas concentration data; and preprocess the multimodal monitoring data stream to obtain standardized thermal imaging images, standardized visible light video frames, and standardized time-series data; S2) Use the ResNet50 model to extract features from the standardized thermal image to obtain the thermal imaging feature vector; S3) Extract visible light feature vectors from standardized visible light video frames using the MobileNetV3 model; S4) Use a 1D-CNN model to encode the features of standardized time-series data and extract time-series feature vectors; S5) Jointly encode and dynamically weight fused the thermal imaging feature vector, visible light feature vector, and time series feature vector to obtain a multimodal fusion feature vector; S6) Using the Transformer time-series prediction model, the current multimodal fusion feature vector and historical multimodal fusion feature sequence are predicted, and the identification results and corresponding confidence levels of four types of precursors, namely initial bulging, abnormal temperature rise, smoke generation and slight leakage, are output, as well as the battery state change trend, abnormal development speed and risk probability in the next 5 to 30 minutes. S7) Determine the risk level of the storage unit based on the identification and prediction results, and push the warning information to the monitoring user.

[0008] Preferably, in step S2), an SE-Net channel attention module and a CBAM spatial attention module are added to the original ResNet50 model. The SE-Net channel attention module calculates the importance weights of each feature channel through a squeeze-excitation operation. The CBAM spatial attention module generates a spatial attention map by performing channel pooling and spatial convolution on the feature map, thereby enhancing the feature response of hot spots, bulges, and leakage traces on the battery surface.

[0009] Preferably, in step S2), multi-scale thermal imaging feature maps are extracted from the standardized thermal imaging map through multiple convolutional blocks of the improved ResNet50 model; then, the multi-scale thermal imaging feature maps are channel-weighted through the SE-Net channel attention module, spatially weighted through the CBAM spatial attention module, and finally output as thermal imaging feature vectors through a global average pooling layer and a fully connected layer.

[0010] Preferably, in step S3), the MobileNetV3 model is improved by adding an SE-Net channel attention module and a CBAM spatial attention module between the depthwise separable convolutional block and the pooling layer. The basic features of the battery appearance texture and contour are extracted from standardized visible light video frames by using depthwise separable convolutional blocks. The SE-Net channel attention module then weights and enhances key feature channels such as bulge contour, smoke grayscale, and leakage traces. The CBAM spatial attention module further strengthens the feature response of the bulge protrusion area and leakage trace area and suppresses background interference. Finally, the extracted features are mapped to visible light feature vectors through a global average pooling layer and a fully connected layer.

[0011] Preferably, in step S6), the Transformer temporal prediction model includes a Transformer encoder, a multi-label classifier, and a Transformer decoder; wherein, the Transformer encoder includes multiple self-attention layers for learning the global association and temporal dependency of multimodal fusion features to obtain global encoder features; the multi-label classifier includes multiple connection layers for outputting the identification results and corresponding confidence levels of four types of thermal runaway precursors—initial bulging, abnormal temperature rise, smoke generation, and slight leakage—based on the global encoder feature mapping; The Transformer decoder includes multiple cross-attention layers, which receive encoder features and historical multimodal fusion feature sequences output by the Transformer encoder, learn long-term temporal dependencies, and output the battery state change trend, anomaly development speed, and risk occurrence probability within the next 5 to 30 minutes.

[0012] Secondly, the present invention provides a battery storage monitoring system based on deep learning, comprising: The data acquisition module is used to acquire multimodal monitoring data streams, including thermal imaging image sequences of the battery storage area, visible light video frame sequences, ambient temperature and humidity data, and ambient VOC gas concentration data; and to preprocess the multimodal monitoring data streams to obtain standardized thermal imaging images, standardized visible light video frames, and standardized time-series data. The thermal imaging feature extraction module calls the ResNet50 model to extract features from the standardized thermal image and obtain thermal imaging feature vectors. The visible light feature extraction module calls the MobileNetV3 model to extract visible light feature vectors from standardized visible light video frames; The temporal feature extraction module calls a 1D-CNN model to encode features in standardized temporal data and extract temporal feature vectors. The feature fusion module is used to jointly encode and dynamically weight the thermal imaging feature vector, visible light feature vector, and time series feature vector to obtain a multimodal fusion feature vector. The thermal runaway precursor identification and timing prediction module calls the Transformer timing prediction model to predict the current multimodal fusion feature vector and the historical multimodal fusion feature sequence, and outputs the identification results and corresponding confidence levels of four types of precursors: initial bulging, abnormal temperature rise, smoke generation, and slight leakage, as well as the battery state change trend, abnormal development speed and risk probability in the next 5 to 30 minutes. The early warning decision module determines the risk level of the storage unit based on the identification and prediction results, and pushes the early warning information to the monitoring users.

[0013] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the battery storage monitoring method.

[0014] The beneficial technical effects of this invention are as follows: 1. This invention extracts thermal imaging features, visible light features, and temporal features using the ResNet50 model, MobileNetV3 model, and 1D-CNN model, respectively. It then performs weighted fusion through cross-modal attention to achieve dynamic weighted fusion of image features and temporal features, fully exploring the correlation between different types of data, thereby improving the accuracy of recognition and prediction. 2. The Transformer time-series prediction model of this invention realizes the identification of thermal runaway precursors and the prediction of time-series trends. It learns the long-term time-series dependencies of multimodal data and can predict the abnormal change trends and risk probabilities in the next 5 to 30 minutes in advance. The early warning lead time reaches 15 to 25 minutes, which greatly increases the time left for staff to deal with the situation and effectively avoids the expansion of risks. 3. This invention can comprehensively capture various precursory features of battery thermal runaway by monitoring data from multiple dimensions. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the method of an embodiment of the present invention; Figure 2 This is a structural framework diagram of the deep learning architecture of the method in the embodiments of the present invention; Figure 3 This is a schematic diagram of the framework structure of the system according to an embodiment of the present invention. Detailed Implementation

[0016] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings: Example 1 like Figure 1 and 2 As shown, this embodiment provides a battery storage monitoring method based on deep learning, including the following steps: S1) Acquire a multimodal monitoring data stream including thermal imaging image sequence of the battery storage area, visible light video frame sequence, ambient temperature and humidity data, and ambient VOC gas concentration data; and preprocess the multimodal monitoring data stream to obtain standardized thermal imaging images, standardized visible light video frames, and standardized time-series data; In this embodiment, an infrared thermal imaging camera is used to acquire a sequence of thermal images of the battery storage area; a visible light camera is used to acquire a sequence of visible light video frames of the battery storage area; a temperature and humidity sensor is used to acquire ambient temperature and humidity data; and a VOC gas sensor is used to acquire ambient VOC gas concentration data. Specifically, the infrared thermal imaging camera has a resolution of at least 640×512, a detection band of 8~14μm, and a frame rate of 10~20fps; the visible light camera has a resolution of at least 1080P and a frame rate of 20~30fps; the temperature and humidity sensor has a sampling frequency of 1 time / second, a temperature measurement range of -20℃~80℃, and a humidity measurement range of 0~100%RH; and the VOC gas sensor has a sampling frequency of 1 time / second and a detection range of 0~1000ppm.

[0017] In this embodiment, the preprocessing of the multimodal monitoring data stream specifically includes: The acquired thermal imaging sequence was subjected to Gaussian denoising, histogram equalization, temperature calibration and size normalization to obtain standardized thermal imaging images. The acquired visible light video frame sequence was subjected to Gaussian denoising, image enhancement, target region cropping, and size normalization to obtain standardized visible light video frames. Outliers were removed from the collected temperature and humidity data and VOC gas concentration data using the Raida criterion, missing values ​​were filled in using linear interpolation, and the data were normalized to the [0,1] interval using the max-min normalization method to obtain standardized temperature and humidity data and VOC gas concentration data. The standardized temperature and humidity data and VOC gas concentration data were then concatenated to obtain standardized time series data.

[0018] S2) Use the ResNet50 model to extract features from the standardized thermal image to obtain the thermal imaging feature vector; In this embodiment, an SE-Net channel attention module and a CBAM spatial attention module are added to the original ResNet50 model. The SE-Net channel attention module calculates the importance weights of each feature channel through a squeeze-excitation operation. The CBAM spatial attention module generates a spatial attention map by performing channel pooling and spatial convolution on the feature map, thereby enhancing the feature responses of hot spots, bulges, and leakage traces on the battery surface.

[0019] In this embodiment, the standardized thermal image is input into multiple convolutional blocks through the input layer of the improved ResNet50 model to extract features, resulting in a multi-scale thermal imaging feature map. Subsequently, the multi-scale thermal imaging feature map is channel-weighted through the SE-Net channel attention module, and then spatially weighted through the CBAM spatial attention module. Finally, it passes through a global average pooling layer and a fully connected layer to output a thermal imaging feature vector. The thermal imaging feature vector includes battery surface temperature gradient features, hotspot coordinate features, temperature extreme value features, and heat diffusion direction features.

[0020] S3) Extract visible light feature vectors from standardized visible light video frames using the MobileNetV3 model; In this embodiment, the MobileNetV3 model is improved by adding an SE-Net channel attention module and a CBAM spatial attention module between the depthwise separable convolutional blocks and pooling layers. After inputting standardized visible light video frames, the basic features of the battery's appearance texture and contour are extracted through depthwise separable convolutional blocks. The SE-Net channel attention module then weights and enhances key feature channels such as bulge contour, smoke grayscale, and leakage traces. The CBAM spatial attention module further strengthens the feature responses of the bulge protrusion area and leakage trace area, suppressing background interference. Finally, a global average pooling layer and a fully connected layer map the extracted features into visible light feature vectors. The visible light feature vectors include battery appearance texture features, bulge contour features, smoke grayscale features, and leakage trace features.

[0021] S4) Use a 1D-CNN model to encode the features of standardized time-series data and extract time-series feature vectors; In this embodiment, the 1D-CNN model includes multiple convolutional layers, multiple pooling layers, fully connected layers, and a ReLU activation function; the temporal feature vector is extracted using the 1D-CNN model as follows: First, short-term local features are extracted from standardized time-series data through the first convolutional layer. After batch normalization and ReLU activation, the short-term local features are input into the first pooling layer to reduce the feature dimension and retain key local features, thus obtaining short-term key local features. Subsequently, the short-term local key features are input into the second convolutional layer to extract the medium-term correlation features. After passing through the ReLU activation function and the second pooling layer, the medium-term correlation key features are obtained. The key features of intermediate correlation are input into the third convolutional layer to extract long-term global features. Finally, the features are mapped to a fixed-length feature vector through a global average pooling layer, and a temporal feature vector is obtained through a fully connected layer. The temporal feature vector includes temporal statistical features, temporal change features, and temporal correlation features.

[0022] S5) Jointly encode and dynamically weight fused the thermal imaging feature vector, visible light feature vector, and time series feature vector to obtain a multimodal fused feature vector; specifically as follows: S51) Through two independent fully connected layers, the thermal imaging feature vector and the visible light feature vector are mapped to the same dimension as the temporal feature vector, respectively; S52) Calculate the channel attention weights for thermal imaging feature vectors and visible light feature vectors. ,Right now: In the formula, Channel weight; Use the Sigmoid activation function; It is the ReLU activation function; , The height and width of the feature map; , This is the weight matrix of the activation layer; For input features; S53) Calculate the spatial attention weights of thermal imaging feature vectors and visible light feature vectors. ;Right now: In the formula, Conv represents the convolution operation; C represents the number of feature channels; Let c be the input feature of the c-th channel; S54), Calculate the temporal attention weights, i.e.: In the formula, For the first Temporal attention weights for each time step; For the first Temporal characteristics of each time step; This represents the total number of time steps. S55) Based on channel attention weights, spatial attention weights, and temporal attention weights, the thermal imaging feature vector, visible light feature vector, and temporal feature vector are weighted and fused to obtain a multimodal fused feature vector, namely: In the formula, This is a multimodal fusion feature vector; , These are thermal imaging feature vectors. Channel attention weights and spatial attention weights; , These are the visible light feature vectors. Channel attention weights and spatial attention weights; For temporal attention weights; This is a time-series feature vector.

[0023] S6) Using the Transformer time-series prediction model, the current multimodal fusion feature vector and historical multimodal fusion feature sequence are predicted, and the identification results and corresponding confidence levels of four types of precursors, namely initial bulging, abnormal temperature rise, smoke generation and slight leakage, are output, as well as the battery state change trend, abnormal development speed and risk probability in the next 5 to 30 minutes. In this embodiment, the Transformer temporal prediction model includes a Transformer encoder, a multi-label classifier, and a Transformer decoder. The Transformer encoder includes multiple self-attention layers to learn the global association and temporal dependencies of multimodal fusion features, obtaining global encoder features. The multi-label classifier includes multiple connection layers to output the identification results and corresponding confidence levels of four types of thermal runaway precursors—initial bulging, abnormal temperature rise, smoke generation, and minor leakage—based on the global encoder feature mapping. The Transformer decoder includes multiple cross-attention layers to receive the encoder features output by the Transformer encoder and historical multimodal fusion feature sequences, learn long-term temporal dependencies, and output the battery state change trend, abnormal development speed, and risk occurrence probability within the next 5-30 minutes.

[0024] In this embodiment, the current multimodal fusion feature vector is concatenated with the historical multimodal fusion feature sequence to obtain the input sequence. ,Right now: ; In the formula, To adjust the sliding window size, This represents the current multimodal fusion feature vector.

[0025] The normalized input sequence The input Transformer encoder employs a multi-head self-attention, feedforward network, and residual connection structure for each self-attention layer; specifically as follows: The normalized input sequence is first processed through multiple connection layers. They are mapped to query vector Q, key vector K, and value vector V, respectively; and the query vector Q, key vector K, and value vector V are further divided into sub-vectors corresponding to multiple attention heads according to the channel dimension. , ; Each attention head independently performs the scaled dot product attention operation, resulting in the output of each attention head. ,Right now: In the formula, For activation functions; To scale attention scores; Then, the outputs of multiple self-attention heads are concatenated and dimensionally integrated through a fully connected layer to obtain multi-head attention features. Then the input is fed forward into a network to enhance the feature representation, that is: ; In the formula, This indicates a feedforward reinforcement feature; , Here is the weight matrix of the feedforward network; , For feedforward network bias terms; Residual connections are added after each self-attention layer and feedforward network. After layer normalization, the layer features of each self-attention layer are obtained. After multiple self-attention layers, the final global encoder features are obtained. .

[0026] In this embodiment, the multi-label classifier processes global encoder features through multiple fully connected layers. Feature mapping is performed, progressively compressing the feature dimensions, and finally outputting the mapped feature vector, i.e.: ; ; ; In the formula, This is the final mapped feature vector; , , These are the weight matrices for the three fully connected layers; , , These are the bias terms for the three fully connected layers; , These are the mapping feature vectors of the first and second fully connected layers, respectively. Then, the final mapped feature vector is obtained by applying the sigmoid activation function. Mapped to confidence levels of four types of precursors ,Right now: In the formula, For the first Precursor-like symptoms ( The confidence levels correspond to initial bulging, abnormal temperature rise, smoke generation, and minor leakage, respectively. For the final mapped feature vector The One element; Finally, by setting a confidence threshold ,when Determine the first If the precursor exists, then the precursor is considered not to exist; otherwise, it is determined that the precursor does not exist. ; Label Indicates the first If a precursor to the class exists, otherwise, the first There are no precursors to this type of disease.

[0027] In this embodiment, a weighted sum of the cross-entropy loss function and the Focal Loss loss function is used to solve the class imbalance problem in multi-label classification, that is: ; In the formula, To identify the loss function; The cross-entropy loss function; Focal Loss is the loss function. These are the weighting coefficients; The aforementioned cross-entropy loss function Represented as: The Focal Loss function is expressed as follows: In the formula, The number of samples; For the first The first sample Tags for early warning signs; For the first The first sample Confidence level of similar precursors; Category weights; For focusing parameters.

[0028] In this embodiment, the Transformer decoder uses global encoder features. Using the target sequence as a mask input and within the given context, the output shows the battery state change trend for the next 5-30 minutes, as follows: Set the prediction time step to M and initialize the target sequence. ; Each cross-attention layer of the Transformer decoder is a structure consisting of masked multi-head self-attention, cross-attention, feedforward network, and residual connection. First, information from future time steps is masked using multi-head self-attention masking, and then cross-attention is used to combine the contextual features output by the Transformer encoder; that is: In the formula, This represents the result of multi-head self-attention calculation for the mask; This represents the result of cross-attention calculation; , , The query, key, and value vectors mapping to the target sequence Y; , Global encoder features for Transformer encoders The key-value vector of the mapping; This is a lower triangular mask matrix used to mask information from future time steps, ensuring the autoregressive nature of the prediction.

[0029] The predicted sequence is obtained by enhancing the features of the final output of cross-attention through a feedforward network, then normalizing it through residual connections and layers, and finally iterating through multiple layers of cross-attention. ; Then predict the sequence The fully connected layer maps to specific prediction metrics, including changes in the height of the bulge. Rate of temperature rise VOC gas concentration changes Probability of risk occurrence ;Right now: In the formula, , These are the predicted weight matrix and bias term, respectively; The feature vector at time step t of the predicted sequence; probability of risk occurrence. Normalized to the [0,1] interval by the sigmoid function, it is used as the probability of risk occurrence.

[0030] In this embodiment, the mean squared error loss function is used as the prediction loss function. It measures the deviation between the predicted value and the actual value, that is: In the formula, M is the set prediction time step; For the first The first time step Predicted values ​​for each predictive indicator; For the first The first time step The true value of each predictive indicator.

[0031] In this embodiment, the total loss function of the Transformer time series prediction model is... To identify the weighted sum of the loss function and the prediction loss function, i.e.: ; Through the total loss function Parameter updates during Transformer time series prediction model training: The Adam optimizer is used to minimize the total loss function and update all weight parameters of the model to ensure recognition and prediction accuracy.

[0032] S7) Determine the risk level of the storage unit based on the identification and prediction results, and push the warning information to the monitoring user.

[0033] In this embodiment, the risk levels include: Level of concern: There is a single low-confidence precursor, and the probability of future risk is ≤30%. Warning level: There is a single high-confidence precursor or multiple low-confidence precursors, with a future risk probability of 30% to 70%. Alert level: Multiple high-confidence precursors exist, with a future risk probability >70%.

[0034] Example 2 like Figure 3 As shown, this embodiment provides a battery warehouse monitoring system based on deep learning, including: The data acquisition module is used to acquire multimodal monitoring data streams, including thermal imaging image sequences of the battery storage area, visible light video frame sequences, ambient temperature and humidity data, and ambient VOC gas concentration data; and to preprocess the multimodal monitoring data streams to obtain standardized thermal imaging images, standardized visible light video frames, and standardized time-series data. The thermal imaging feature extraction module calls the pre-trained ResNet50 model to extract features from the standardized thermal image and obtain the thermal imaging feature vector. The visible light feature extraction module calls the MobileNetV3 model to extract visible light feature vectors from standardized visible light video frames; The temporal feature extraction module calls a 1D-CNN model to encode features in standardized temporal data and extract temporal feature vectors. The feature fusion module is used to jointly encode and dynamically weight the thermal imaging feature vector, visible light feature vector, and time series feature vector to obtain a multimodal fusion feature vector. The thermal runaway precursor identification and timing prediction module calls the Transformer timing prediction model to predict the current multimodal fusion feature vector and the historical multimodal fusion feature sequence, and outputs the identification results and corresponding confidence levels of four types of precursors: initial bulging, abnormal temperature rise, smoke generation, and slight leakage, as well as the battery state change trend, abnormal development speed and risk probability in the next 5 to 30 minutes. The early warning decision module determines the risk level of the storage unit based on the identification and prediction results, and pushes the early warning information to the monitoring users.

[0035] Example 3 This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the battery storage monitoring method described in Embodiment 1.

[0036] In this embodiment, the memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. A processor, coupled to the memory, is used to execute computer programs stored in the memory.

[0037] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form.

[0038] The embodiments and descriptions above are merely illustrative of the principles and preferred embodiments of the present invention. Various changes and modifications may be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.

Claims

1. A battery storage monitoring method based on deep learning, characterized in that, Includes the following steps: S1) Acquire a multimodal monitoring data stream including thermal imaging image sequence of the battery storage area, visible light video frame sequence, ambient temperature and humidity data, and ambient VOC gas concentration data; and preprocess the multimodal monitoring data stream to obtain standardized thermal imaging images, standardized visible light video frames, and standardized time-series data; S2) Use the ResNet50 model to extract features from the standardized thermal image to obtain the thermal imaging feature vector; S3) Extract visible light feature vectors from standardized visible light video frames using the MobileNetV3 model; S4) Use a 1D-CNN model to encode the features of standardized time series data and extract time series feature vectors; S5) Jointly encode and dynamically weight fused the thermal imaging feature vector, visible light feature vector, and time series feature vector to obtain a multimodal fusion feature vector; S6) Using the Transformer time-series prediction model, the current multimodal fusion feature vector and historical multimodal fusion feature sequence are predicted, and the identification results and corresponding confidence levels of four types of precursors, namely initial bulging, abnormal temperature rise, smoke generation and slight leakage, are output, as well as the battery state change trend, abnormal development speed and risk probability in the next 5 to 30 minutes. S7) Determine the risk level of the corresponding storage unit based on the identification and prediction results, and push the early warning information to the monitoring user.

2. The battery storage monitoring method based on deep learning according to claim 1, characterized in that, In step S2), the ResNet50 model is improved by adding an SE-Net channel attention module and a CBAM spatial attention module to the original ResNet50 model. The SE-Net channel attention module calculates the importance weight of each feature channel through a squeeze-excitation operation. The CBAM spatial attention module generates a spatial attention map by performing channel pooling and spatial convolution on the feature map, thereby enhancing the feature response of hot spots, bulges, and leakage traces on the battery surface.

3. The battery storage monitoring method based on deep learning according to claim 2, characterized in that, In step S2), multi-scale thermal imaging feature maps are extracted from the standardized thermal imaging map through multiple convolutional blocks of the improved ResNet50 model; then, the multi-scale thermal imaging feature maps are channel-weighted by the SE-Net channel attention module, spatially weighted by the CBAM spatial attention module, and finally output as thermal imaging feature vectors after passing through a global average pooling layer and a fully connected layer.

4. The battery storage monitoring method based on deep learning according to claim 1, characterized in that, In step S3), the MobileNetV3 model is improved by adding SE-Net channel attention module and CBAM spatial attention module between the depthwise separable convolutional block and pooling layer. The basic features of the battery appearance texture and contour are extracted from standardized visible light video frames by using depthwise separable convolutional blocks. The SE-Net channel attention module then weights and enhances key feature channels such as bulge contour, smoke grayscale, and leakage traces. The CBAM spatial attention module further strengthens the feature response of the bulge protrusion area and leakage trace area and suppresses background interference. Finally, the extracted features are mapped to visible light feature vectors through a global average pooling layer and a fully connected layer.

5. The battery storage monitoring method based on deep learning according to claim 1, characterized in that, In step S5), the thermal imaging feature vector, visible light feature vector, and temporal feature vector are weighted and fused based on channel attention weight, spatial attention weight, and temporal attention weight to obtain a multimodal fusion feature vector.

6. The battery storage monitoring method based on deep learning according to claim 1, characterized in that, In step S6), the Transformer temporal prediction model includes a Transformer encoder, a multi-label classifier, and a Transformer decoder; wherein, the Transformer encoder includes multiple self-attention layers, used to learn the global association and temporal dependency of multimodal fusion features to obtain global encoder features; the multi-label classifier includes multiple connection layers, used to output the identification results and corresponding confidence levels of four types of thermal runaway precursors, namely initial bulging, abnormal temperature rise, smoke generation, and slight leakage, based on the global encoder feature mapping; The Transformer decoder includes multiple cross-attention layers, which receive encoder features and historical multimodal fusion feature sequences output by the Transformer encoder, learn long-term temporal dependencies, and output the battery state change trend, anomaly development speed, and risk occurrence probability within the next 5 to 30 minutes.

7. The battery storage monitoring method based on deep learning according to claim 6, characterized in that, In step S6), the multi-label classifier processes the global encoder features through multiple fully connected layers. Perform feature mapping, gradually compress the feature dimensions, and finally output the mapped feature vector; Then, the final mapped feature vector is mapped to the confidence scores of four types of precursors using the sigmoid activation function; Finally, by setting a confidence threshold, if the confidence of the precursor is greater than the confidence threshold, the precursor is determined to exist; otherwise, the precursor is determined not to exist. Furthermore, a weighted sum of the cross-entropy loss function and the Focal Loss loss function is adopted to solve the class imbalance problem in multi-label classification.

8. The battery storage monitoring method based on deep learning according to claim 7, characterized in that, In step S6), the Transformer decoder, using the global encoder features as context and combining them with the mask input of the target sequence, outputs the battery state change trend for the next 5-30 minutes, as follows: Set the prediction time step and initialize the target sequence; Each cross-attention layer of the Transformer decoder is a structure consisting of masked multi-head self-attention, cross-attention, feedforward network, and residual connection. First, information about future time steps is masked by multi-head self-attention through masking, and then the contextual features output by the Transformer encoder are combined with cross-attention. The predicted sequence is obtained by passing the final output of cross attention through a feedforward network to enhance features, then through residual connections and layer normalization, and finally through multiple layers of cross attention. The predicted sequences are then mapped to specific prediction metrics through a fully connected layer, including changes in bulge height, rate of temperature rise, changes in VOC gas concentration, and probability of risk occurrence.

9. A battery storage monitoring system based on deep learning, characterized in that, Includes the following modules: The data acquisition module is used to acquire multimodal monitoring data streams, including thermal imaging image sequences of the battery storage area, visible light video frame sequences, ambient temperature and humidity data, and ambient VOC gas concentration data; and to preprocess the multimodal monitoring data streams to obtain standardized thermal imaging images, standardized visible light video frames, and standardized time-series data. The thermal imaging feature extraction module calls the ResNet50 model to extract features from the standardized thermal image and obtain thermal imaging feature vectors. The visible light feature extraction module calls the MobileNetV3 model to extract visible light feature vectors from standardized visible light video frames; The temporal feature extraction module calls a 1D-CNN model to encode features in standardized temporal data and extract temporal feature vectors. The feature fusion module is used to jointly encode and dynamically weight the thermal imaging feature vector, visible light feature vector, and time series feature vector to obtain a multimodal fusion feature vector. The thermal runaway precursor identification and timing prediction module calls the Transformer timing prediction model to predict the current multimodal fusion feature vector and the historical multimodal fusion feature sequence, and outputs the identification results and corresponding confidence levels of four types of precursors: initial bulging, abnormal temperature rise, smoke generation, and slight leakage, as well as the battery state change trend, abnormal development speed and risk probability in the next 5 to 30 minutes. The early warning decision module determines the risk level of the corresponding storage unit based on the identification and prediction results, and pushes the early warning information to the monitoring users.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the battery storage monitoring method as described in any one of claims 1-8.

Images