Energy storage system image recognition and fault early warning system
By synchronously acquiring the time-series video stream and vibration acoustic signal of the energy storage system, a cross-modal association dictionary and virtual sample library are constructed, and the cross-modal recognition model is optimized. This solves the problem of high false alarm rate in energy storage systems and enables accurate identification and early warning of fault characteristics.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 上海汇祯实业有限公司
- Filing Date
- 2026-04-15
- Publication Date
- 2026-05-29
Smart Images
Figure CN122116242A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition and fault detection technology, and discloses an image recognition and fault early warning system based on energy storage systems. Background Technology
[0002] Current image recognition and fault early warning solutions for energy storage systems primarily rely on static single-frame image detection technology. In practice, cameras are typically deployed fixedly outside the energy storage cabinet to capture images at a single point in time. Then, convolutional neural networks are used to extract spatial features such as edges and textures from the images, which are then compared pixel-by-pixel with a pre-set template of normal appearance or a known library of typical fault images to determine the presence of surface cracks, corrosion, or significant deformation. For monitoring the internal structure, this often relies on the analysis of single-type sensor signals, such as acquiring vibration signals and extracting their frequency domain amplitude, setting fixed thresholds for over-limit alarms. Such solutions only perform slice-like analysis of the physical state at a specific instant, outputting alarm results based on static spatial characteristics.
[0003] The aforementioned existing technologies, when monitoring the entire charge-discharge cycle of energy storage systems, suffer from a high false alarm rate due to their inability to distinguish between dynamic characteristics under normal operating conditions and abnormal fault characteristics. Because energy storage cells undergo normal physical expansion and contraction during charge-discharge cycles, and components such as circuit breakers and cooling fans exhibit normal mechanical movements, existing detection schemes based on static single-frame images may misjudge dynamic deformations and component operation during normal charge-discharge processes as structural damage or mechanical faults. Furthermore, single-sensor threshold alarms cannot reflect the continuous evolution of physical characteristics over time, leading to frequent false alarms triggered by fluctuations in normal operating conditions. Summary of the Invention
[0004] The purpose of this invention is to provide an image recognition and fault early warning system based on energy storage systems, thereby solving the problems mentioned in the background art.
[0005] An image recognition and fault early warning system based on energy storage systems includes: Simultaneously acquire time-series video streams and vibration acoustic signals throughout the entire charging and discharging cycle of the energy storage system; The dynamic deformation features and component movement features of the battery cell and module are extracted from the video stream by a time-series motion localization network, and the normal expansion and abnormal bulging deformation, and normal movement and abnormal jamming are distinguished. Modal features of the vibration acoustic signal are extracted, and a cross-modal association dictionary of visual dynamic features and vibration acoustic features is constructed. Based on digital twins, virtual samples of different fault types and degradation levels are generated. Combined with small sample incremental meta-learning algorithm, the cross-modal recognition model is optimized using virtual samples and actual test samples. The system compares measured features with standard features of the digital twin in real time, and outputs fault warning signals, fault levels, and fault evolution trend prediction results.
[0006] Preferably, the dynamic deformation features and component motion features of the battery cells and modules in the video stream are extracted, including: The energy storage cabinet in the time-series video stream is divided into multiple grid nodes, and a spatiotemporal topology graph containing the physical coordinates of the nodes and their adjacency relationships is constructed. The pixel temporal variation sequence of each grid node is used as a node attribute, and the node state is propagated along the topological edges using a graph convolutional network; Manifold learning is introduced to reduce the dimensionality of the high-dimensional spatiotemporal features output by the graph convolutional network, and extract low-dimensional manifold feature vectors that reflect the continuous evolution of cell deformation and component movement in physical space.
[0007] Preferably, the modal features extracted from the vibration acoustic signal include: Phase space reconstruction is performed on the vibration acoustic signal, and the correlation dimension and maximum Lyapunov exponent of the trajectory in the reconstructed phase space are calculated to obtain chaotic features characterizing the nonlinear evolution of early weak faults. The chaotic features are fused with the Mel frequency cepstral coefficients of the acoustic signal in series. Nonnegative matrix factorization is used to decouple the fused features, separating the main modal features reflecting the resonance of the structural body and the residual features reflecting environmental noise interference. The main modal features are used as the final modal features.
[0008] Preferably, a cross-modal association dictionary of visual dynamic features and vibroacoustic features is constructed, including: A dual-stream Transformer architecture is used to perform self-attention encoding on visual dynamic features and vibroacoustic modal features respectively; A cross-modal contrastive learning loss function based on Wasserstein distance is introduced to align the edge distributions and conditional distributions of visual and acoustic flows at the feature distribution level. The aligned multimodal joint features are mapped to a sparse coding space, and the L1 norm regularization optimization problem is solved iteratively to generate a robust cross-modal association dictionary that preserves cross-modal causal relationships.
[0009] Preferably, virtual samples with different fault types and degradation levels are generated based on digital twins, including: Construct a physical information neural network with constraints from coupled thermodynamic-electrochemical-structural mechanics partial differential equations; Using the operating boundary conditions and initial state of the energy storage system as input, virtual operating state parameters are output through forward propagation. By coupling the data loss term and the partial differential equation residual term in the loss function of the physical information neural network, the generated virtual samples strictly follow the multi-physics field evolution mechanism of the energy storage system, forming a virtual fault sample library with physical determinism.
[0010] Preferably, a graph convolutional network is used to propagate node states along the topological edges, including: A dynamic network routing mechanism is introduced to calculate the information transmission gain of each topological edge at the current moment and to globally sort the information transmission gains of all topological edges. Eliminate a predetermined proportion of topological edge state propagation paths that are ranked lower in the sorting, and suppress redundant information interaction between nodes under normal operating conditions. For the retained topological edges, differentiated attention weights are assigned based on the gain value. When a sudden change in the topological structure is detected, long-range skip connections are activated to amplify the weight of state transfer from the abnormally deformed region to the surrounding nodes, thereby achieving adaptive focusing of computing power on the potential fault region.
[0011] Preferably, before performing phase space reconstruction on the vibration acoustic signal, the method further includes: Adaptive wavelet packet decomposition is used to divide the original vibroacoustic signal into multiple scales; Using the energy entropy of each sub-band as an individual in the population, a frequency band optimization model based on a genetic algorithm is constructed. The fitness function is to maximize the fault feature distinguishability, and the optimal combination of sub-bands containing the most early fault singular features is selected. Phase space reconstruction is performed only on the selected optimal sub-band combinations to filter out broadband background interference unrelated to faults.
[0012] Preferably, generating a cross-modal association dictionary further includes: The cross-modal association dictionary in the sparse coding space is represented as a high-order tensor, where each dimension of the tensor corresponds to a different fault type, time scale, and modal source. Tensor train decomposition network is used to perform rank-constrained dimensionality reduction on high-order tensors, decomposing the original high-dimensional tensor into a series of low-dimensional core tensors and factor matrices. During the fault warning process, the cross-modal association dictionary is recovered by forward propagation of the low-dimensional core tensor and factor matrix, which reduces the computational complexity and memory usage of multimodal feature joint retrieval.
[0013] Preferably, the cross-modal recognition model is optimized by combining a few-sample incremental meta-learning algorithm with virtual samples and actual test samples, including: An active learning strategy is introduced into the physically deterministic virtual fault sample library to calculate the prediction entropy and information density of each virtual sample under the current cross-modal recognition model parameters. Boundary support samples are selected based on the joint distribution of prediction entropy and information density; The selected boundary support samples and a small number of measured samples are used to construct a meta-task. By optimizing the parameters in the inner and outer loop training of the meta-task, the cross-modal recognition model is forced to learn how to quickly fine-tune on a small number of measured samples of new energy storage systems.
[0014] Preferably, this forces the cross-modal recognition model to learn how to rapidly fine-tune on a small number of measured samples of new energy storage systems, including: A federated learning framework across energy storage power stations is adopted to perform meta-tasks and inner and outer loop parameter updates based on boundary support samples locally at each power station. In the cloud aggregation node, a topology adaptive weighting algorithm is introduced to dynamically allocate aggregation weights based on the cosine similarity between the local model update gradient uploaded by each power station and the global model gradient, thereby alleviating model drift caused by the non-independent and identically distributed data distribution of different types of energy storage systems. The aggregated global model serves as the initialization parameter for the next local meta-learning, enabling cross-power station privacy protection and continuous evolution of fault early warning capabilities.
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention, by simultaneously acquiring time-series video streams and vibratory acoustic signals and extracting a cross-modal association dictionary, can distinguish between the dynamic features of normal operating conditions and abnormal fault features of energy storage systems, reducing the false alarm rate. The system employs graph convolutional networks and manifold learning to extract low-dimensional manifold feature vectors of the continuous evolution of cells and components in physical space within the video stream. Combined with phase space reconstruction, it obtains the nonlinear evolution chaotic features of the vibratory acoustic signals, enabling the model to track continuous changes in physical state over time. By using a dual-stream attention architecture and a bulldozer distance-based comparative learning loss function, the visual and acoustic streams are aligned at the feature distribution level, generating a cross-modal association dictionary in a sparse coding space. This transforms static spatial features into joint features containing temporal evolution logic and cross-modal causal relationships. Based on this dynamic evolution law, comparisons are made, eliminating interference from normal charging and discharging expansion and conventional component actions, and identifying abnormal bulging deformation and abnormal jamming.
[0016] 2. This invention employs a physical information neural network and a small-sample incremental meta-learning algorithm to reduce the dependence of the cross-modal identification model on measured labeled samples. By coupling the residual terms of partial differential equations into the loss function, a virtual fault sample library conforming to the multi-physics evolution mechanism is generated. Combined with an active learning strategy, boundary support samples are selected to construct meta-tasks, enabling the model to update parameters on a small number of measured samples from new energy storage systems. By combining a federated learning framework across energy storage power stations with a topology adaptive weighting algorithm, aggregation weights are dynamically allocated based on gradient cosine similarity to alleviate model drift caused by differences in data distribution among different energy storage systems, enabling parameter updates of the early warning model across different power stations. Simultaneously, the cross-modal association dictionary is represented as a high-order tensor and dimensionality is reduced using a tensor train decomposition network. The dictionary is recovered by forward propagation of the low-dimensional core tensor and factor matrix, reducing the computational complexity and memory consumption of multi-modal feature joint retrieval. Attached Figure Description
[0017] Figure 1 This is a flowchart illustrating the overall process of fault early warning for energy storage systems based on image recognition and multimodal fusion, as described in this invention. Figure 2 This is a flowchart of the spatiotemporal topology graph convolution + manifold learning visual dynamic feature extraction process of the present invention; Figure 3 This is a flowchart of the adaptive wavelet packet + chaotic feature decoupling acoustic mode extraction of the present invention; Figure 4 This is a flowchart of the construction process for the dual-stream Transformer+Wasserstein aligned cross-modal dictionary of the present invention; Figure 5 This is a flowchart of the multi-physics field coupled PINN virtual fault sample generation process of the present invention; Figure 6 This is a flowchart of the optimization process for the active learning meta-learning + federated weighted aggregation model of the present invention. Detailed Implementation
[0018] Example 1: Please refer to Figure 1 The system employs a hardware-triggered synchronous acquisition mechanism, using the charge / discharge cycle start signal output by the energy storage system's BMS as the trigger reference. This mechanism adds a unified timestamp to the time-series video stream and the vibratory acoustic signal, with a synchronization accuracy of at least 1ms. The acquisition frame rate of the time-series video stream is positively correlated with the charge / discharge rate. When the charge / discharge rate is greater than or equal to 1C, the acquisition frame rate is set to 30fps; when the charge / discharge rate is less than 1C, the acquisition frame rate is set to 15fps. The sampling frequency of the vibratory acoustic signal is set to 48kHz, with an acquisition bandwidth covering 20Hz-20kHz, ensuring full-frequency coverage of cell structural deformation and mechanical component operation.
[0019] The system processes synchronously acquired time-series video streams through a time-series motion localization network, extracting dynamic deformation features and component movement features of the battery cells and modules from the video streams, and distinguishing between normal expansion and abnormal bulging deformation, and normal movement and abnormal stagnation. The time-series motion localization network takes a continuous frame sequence of the time-series video stream as input and outputs dynamic feature vectors reflecting the continuous evolution of battery cell deformation and component movement in the time dimension. Through a preset time-series motion classifier, the feature vectors are mapped to four preset categories: normal expansion, abnormal bulging, normal movement, and abnormal stagnation, completing feature extraction and state differentiation.
[0020] The system extracts features from synchronously acquired vibration acoustic signals to obtain modal features characterizing the nonlinear evolution of the signals. After preprocessing the vibration acoustic signals, the system extracts the time-domain, frequency-domain, and nonlinear features of the signals. Through feature decoupling, it separates the effective features related to fault evolution as the final vibration acoustic modal features.
[0021] The system constructs a cross-modal association dictionary of visual dynamic features and vibration-acoustic features. The system performs unified encoding on the visual dynamic features and vibration-acoustic modal features, aligns the distributions of the two types of features at the feature distribution level, maps the aligned multimodal joint features to a sparse coding space, and obtains the cross-modal association dictionary through iterative optimization. This cross-modal association dictionary preserves the causal relationship between visual and acoustic features and is used for subsequent feature matching and fault identification.
[0022] The system generates virtual samples of different fault types and degradation levels based on digital twins. Combining a small-sample incremental meta-learning algorithm, it optimizes the cross-modal identification model using both virtual and measured samples. The digital twin is a digital mapping model embedded with the multi-physics coupling evolution law of the energy storage system. It takes the operating boundary conditions and initial state of the energy storage system as input and outputs standard operating characteristics and fault evolution characteristics under the corresponding operating conditions. The system generates virtual samples covering all fault types and all degradation stages by modifying the fault and degradation parameters of the digital twin, constructing a virtual fault sample library. Using virtual samples as the base samples and field-collected measured samples as adaptation samples, the system optimizes the initialization parameters of the cross-modal identification model using a small-sample incremental meta-learning algorithm, enabling the model to quickly adapt to new energy storage system models with a limited number of measured samples.
[0023] The system compares measured features with standard features of the digital twin in real time, and outputs fault warning signals, fault levels, and fault evolution trend prediction results. The system extracts measured multimodal joint features under the current operating condition in real time, completes sparse encoding and reconstruction of features through a cross-modal association dictionary, and calculates the reconstruction error between the measured features and the standard features output by the digital twin under the same operating condition. When the reconstruction error exceeds a preset warning threshold, the system outputs a warning signal corresponding to the fault type, classifies the fault level based on the magnitude of the reconstruction error and the feature matching degree, and simultaneously uses the multiphysics evolution model of the digital twin to deduce the continuous evolution path of fault parameters, outputting the fault evolution trend and the remaining effective operating time window.
[0024] As a preferred embodiment, refer to Figure 2 When the system extracts the dynamic deformation features and component movement features of the battery cells and modules from the video stream, it first maps the physical space of the energy storage cabinet to the image pixel space for each frame of the time-series video stream using preset camera calibration parameters. The image is then divided into M×N grid nodes, each corresponding to a fixed-size region within the physical space of the energy storage cabinet. The physical coordinates of each node are... ,in Let z be the planar coordinates, and z be the calibration coordinates along the cabinet depth direction. The system constructs topological edges for the grid nodes. The topological edges include spatially adjacent edges and temporally adjacent edges. Spatially adjacent edges connect grid nodes that are spatially adjacent in the same frame image, and temporally adjacent edges connect grid nodes with the same physical coordinates in adjacent frames image. This constructs a spatiotemporal topological graph G=(V,E) containing the physical coordinates and adjacency relationships of the nodes, where V is the set of grid nodes and E is the set of topological edges.
[0025] The system uses the pixel temporal variation sequence of each grid node as a node attribute. For the k-th grid node in the t-th frame image, its pixel temporal variation sequence... for:
[0026] in The preset timing window length corresponds to a preset time segment of the charge / discharge cycle; The first The grid node, at the... The pixel values on the frame image. The system uses a graph convolutional network to propagate node states along the topological edges. The inter-layer propagation calculation formula for the graph convolutional network is:
[0027] in, For the first The node feature matrix of the layer network, For the first The node feature matrix of the layer network, The normalized adjacency matrix, For the first The trainable weight matrix of the layer, It is the ReLU nonlinear activation function.
[0028] When the system uses a graph convolutional network to propagate node states, a dynamic network routing mechanism is introduced to calculate the information transmission gain of each topological edge at the current time. The calculation formula is:
[0029] in, for Time Node eigenvectors, for Time Node eigenvectors, To prevent the minimum value where the denominator is 0, the value is taken as 0. The system globally sorts the information transmission gains of all topological edges, eliminating state propagation paths of edges ranked lower and suppressing redundant information interaction between nodes under normal operating conditions. For the retained topological edges, the system assigns differentiated attention weights based on the gain values. The calculation formula is:
[0030] in, For connecting nodes and nodes Topological edges, This is the set of topological edges to be retained. The system monitors the rate of change of the Frobenius norm of the adjacency matrix in real time. When the rate of change exceeds a preset mutation threshold, it determines that a mutation has occurred in the topological structure, activates long-range skip connections, amplifies the weight of state transfer from the abnormally deformed region to the surrounding nodes, and realizes the adaptive focus of computing power on the potentially faulty region.
[0031] The system introduces manifold learning to reduce the dimensionality of the high-dimensional spatiotemporal features output by the graph convolutional network, extracting low-dimensional manifold feature vectors that reflect the continuous evolution of cell deformation and component movement in physical space. This embodiment employs the isometric mapping (ISOMAP) algorithm to complete the manifold learning for the high-dimensional spatiotemporal feature matrix output by the graph convolutional network. ,in For the set of real numbers, For the number of nodes, To achieve high-dimensional features, the system first calculates the geodesic distance matrix between all node features, constructs a geodesic adjacency graph, and then maps the geodesic distance matrix to a d-dimensional low-dimensional space through multidimensional scaling (MDS). The value is 32, which is much smaller than D, and the low-dimensional manifold eigenvector is finally obtained. , as the final visual dynamic feature.
[0032] As a preferred embodiment, refer to the appendix. Figure 3 Before performing phase space reconstruction on the vibration acoustic signal, the system preprocesses and filters the original vibration acoustic signal. The system employs adaptive wavelet packet decomposition to process the original vibration acoustic signal. conduct Layer wavelet packet decomposition, with j taking a value of 4, yields... Each sub-band has a signal of [number] sub-bands. ,in Sub-band numbering, The system calculates the energy entropy of each sub-band using the following formula:
[0033]
[0034]
[0035] in, For the first Layer Energy of individual frequency bands This refers to the proportion of energy in this sub-band to the total energy. This represents the energy entropy of the sub-band.
[0036] The system uses the energy entropy of each sub-band as an individual in the population to construct a frequency band optimization model based on a genetic algorithm. The fitness function is to maximize the fault feature distinguishability. The calculation formula is:
[0037] in, This represents the average energy entropy of the sub-band combination under fault conditions. This represents the average energy entropy of the sub-band combination under normal conditions. and denoted as the standard deviation of the energy entropy under the corresponding state. The system sets the population size of the genetic algorithm to 50, the crossover probability to 0.7, the mutation probability to 0.01, and the number of iterations to 100. Through selection, crossover, and mutation operations, iterative optimization is performed to select the optimal sub-band combination containing the most early fault singular features. Signal reconstruction and phase space reconstruction are performed only on the selected optimal sub-band combination to filter out broadband background interference unrelated to the fault.
[0038] The system reconstructs the optimal sub-band signal. Phase space reconstruction is performed, and the delay time is determined using the CC method. and embedding dimension The reconstructed phase space trajectory matrix for:
[0039] Represent The component vectors.
[0040] The system calculates the correlation dimension and maximum Lyapunov exponent of the trajectories in the reconstructed phase space to obtain chaotic features characterizing the nonlinear evolution of early weak faults. The correlation dimension is calculated using the GP algorithm, and the formula is as follows:
[0041] in, For the integral of association, Let r be the step function, r be the neighborhood radius, N be the number of points in the phase space, and be the nth step function in the phase space. The, the The phase space state vectors of each phase point have an correlation dimension D2 of . right The slope of the linear segment of the curve. Maximum Lyapunov exponent. The calculation is performed using a small amount of data, and the formula is as follows:
[0042] in, for Time of the first Phase points The nearest neighbor point, for all j, yields the maximum Lyapunov exponent by taking the mean. .
[0043] The system will calculate the chaotic features (correlation dimension D2, maximum Lyapunov exponent) The system performs concatenated fusion with the Mel-frequency cepstral coefficients (MFCCs) of the acoustic signal. The MFCC feature extraction process involves: pre-emphasis, framing, and windowing of the reconstructed signal; obtaining the power spectrum through Fast Fourier Transform; filtering the power spectrum using a Mel-filter bank; taking the logarithm and then performing a Discrete Cosine Transform to extract the first 13 dimensions of MFCC features. The system concatenates the chaotic features with the MFCC features to obtain a fused feature vector. .
[0044] The system uses nonnegative matrix factorization to decouple the fused features. For the fused feature matrix... ,in For the number of time frames, As the feature dimension, the system decomposes it into a basis matrix. sum coefficient matrix ,satisfy and , All elements are non-negative. Given a preset rank of 2, the optimization objective is: ; The system solves the above optimization problem iteratively by using a multiplicative update rule, separating the main modal features reflecting the resonance of the structural body and the residual features reflecting environmental noise interference, and using the main modal features as the final modal features.
[0045] As a preferred embodiment, refer to Figures 4 to 6 When constructing the cross-modal association dictionary of visual dynamic features and vibration-acoustic features, the system employs a dual-stream Transformer architecture to perform self-attention encoding on the visual dynamic features and vibration-acoustic modal features respectively. The dual-stream Transformer architecture includes a visual stream encoder and an acoustic stream encoder, each encoder containing 6 layers of multi-head self-attention layers, with 8 multi-head attention heads and a feedforward network dimension of 2048. The self-attention layers...
[0046] The calculation formula is:
[0047] in, , , These are query, key, and value matrices, respectively. Let be the dimension of the key vector. The low-dimensional manifold feature vector obtained by the visual flow encoder is the input, and the output is the visually encoded feature vector. The principal modal features obtained by the acoustic flow encoder are used as input, and the output is the acoustic encoded features. .
[0048] The system introduces a cross-modal contrastive learning loss function based on Wasserstein distance to align the edge and conditional distributions of the visual and acoustic flows at the feature distribution level. The total loss function... The calculation formula is:
[0049] in, Distribution of visual features Acoustic feature distribution The Wasserstein distance between two classes of features is used to align their marginal distributions. The calculation formula is:
[0050] in, for and The set of all joint distributions, For those following a joint distribution Sample pairs of coordinates Find the expected value. Represents the lower bound.
[0051] The InfoNCE contrastive loss is used to align the conditional distributions of two types of features, and its calculation formula is as follows:
[0052] in, The cosine similarity function is used. It is a temperature coefficient with a value of 0.07. For visual feature vectors, For positive sample acoustic feature vectors, The negative samples are acoustic features, the positive samples are visual-acoustic feature pairs at the same time stamp, and the negative samples are visual-acoustic feature pairs at different time stamps. The system aligns the distributions of visual and acoustic flow features by minimizing the total loss function.
[0053] The system will align the multimodal joint features Mapped to a sparse coding space, an L1 norm regularization optimization problem is iteratively solved to generate a cross-modal association dictionary. The objective function of the optimization problem is:
[0054] in, The cross-modal association dictionary to be solved. It is a sparse coding matrix. The regularization coefficient is 0.1. The system uses the K-SVD algorithm iteratively to solve the above optimization problem. First, the sparse coding matrix is optimized while the dictionary D is fixed. Then, the dictionary is optimized while the sparse coding matrix is fixed. The number of iterations is set to 50. Finally, a robust cross-modal association dictionary that preserves cross-modal causal relationships is generated. This represents the Frobenius norm.
[0055] When generating the cross-modal association dictionary, the system represents the cross-modal association dictionary in the sparse coding space as a high-order tensor form. , Fault type dimension (number of different fault categories); : Time scale dimension (number of scales for multi-scale time series features); Modality source dimension (number of different sensors / modalities). The system uses a tensor train decomposition network to perform rank-constrained dimensionality reduction on high-order tensors, decomposing the original high-dimensional tensor into a series of low-dimensional core tensors multiplied by factor matrices. The calculation formula is:
[0056] in, These are the specific value indices corresponding to fault type, time scale, and modal origin, respectively. , , For low-dimensional core tensors, , , It is rank TT and its value does not exceed 32. During the fault warning process, the system recovers the cross-modal association dictionary by forward propagating the low-dimensional core tensor and factor matrix, thereby reducing the computational complexity and memory consumption of multimodal feature joint retrieval.
[0057] In a preferred embodiment, when the system generates virtual samples of different fault types and degradation levels based on digital twins, it constructs a Physical Information Neural Network (PINN) embedded with constraints from coupled thermodynamic-electrochemical-structural mechanics partial differential equations. The coupled partial differential equations include: Electrochemical solid-phase diffusion equation:
[0058] in, This represents the concentration of solid-phase lithium. The solid-phase diffusion coefficient; Thermodynamic heat conduction equation:
[0059] in, For cell material density, For isobaric specific heat capacity, Thermal conductivity, For cell temperature, This is a heat-generating term, including reversible and irreversible heat generation; Structural mechanics equilibrium equations:
[0060] in, Here, u is the divergence operator, and u is the displacement vector. For stress tensor, For volume forces, the stress tensor and strain tensor satisfy Hooke's law, and the strain is driven by both temperature changes and volume expansion caused by lithium insertion.
[0061] The input to the physical information neural network is the operating boundary conditions and initial state of the energy storage system. The operating boundary conditions include the charging and discharging current I(t) and the ambient temperature. The initial state includes initial lithium concentration distribution, initial temperature distribution, and initial stress distribution. The network outputs virtual operating state parameters through forward propagation, including cell volume expansion, structural vibration response, temperature distribution, SOC, and stress distribution. The loss function of the physical information neural network... for:
[0062] in, The data loss term is the mean square error between the measured operating data and the network output. The partial differential equation residuals are represented by , and the mean square error of the residuals of the coupled partial differential equations is represented by . The system automatically solves for the partial derivatives of the partial differential equations by differentiation, and minimizes the loss function to ensure that the generated virtual samples strictly follow the multi-physics evolution mechanism of the energy storage system. By modifying the fault type parameters and degradation degree parameters input to the network, the system generates virtual samples covering all fault types, such as cell bulging, internal short circuits, loose tabs, and component jamming, as well as the early degradation, mid-term deterioration, and final failure stages, forming a virtual fault sample library with physical determinism.
[0063] The system combines a small-sample incremental meta-learning algorithm with virtual and measured samples to optimize the cross-modal recognition model. An active learning strategy is introduced into the physically deterministic virtual fault sample library to calculate the prediction entropy and information density of each virtual sample under the current cross-modal recognition model parameters. The prediction entropy... The calculation formula is:
[0064] Where C represents the total number of fault types. This represents the predicted class probability of the model for that sample. The information density... The calculation formula is:
[0065] in, The feature vector of the current virtual sample. For the feature vectors of the samples in the training set, This represents the total number of samples in the training set.
[0066] The system selects samples with both predicted entropy and information density exceeding a preset threshold as boundary support samples based on the joint distribution of predicted entropy and information density. These selected boundary support samples, along with a small number of measured samples, are used to construct meta-tasks. Each meta-task consists of a support set and a query set. The support set contains a small number of labeled samples, while the query set is used to evaluate the model update performance. The system employs a Model-Independent Meta-Learning (MAML) algorithm, optimizing parameters through inner and outer loop training within the meta-tasks. The inner loop performs multi-step gradient updates on the model using the support set within a single meta-task, obtaining model parameters adapted to the task. The outer loop updates the model's initialization parameters using the loss from the query set, forcing the cross-modal recognition model to learn how to rapidly fine-tune on a small number of measured samples from new energy storage systems.
[0067] When optimizing the cross-modal recognition model, a federated learning framework across energy storage power stations is adopted. Each power station performs meta-tasks based on boundary support samples, updating parameters in both internal and external loops. Each power station acts as a local node, completing data processing and model parameter updates only locally, without uploading raw operational data; it only uploads the local model update gradients to the cloud aggregation node. At the cloud aggregation node, the system introduces a topology adaptive weighting algorithm, dynamically allocating aggregation weights based on the cosine similarity between the local model update gradient uploaded by each power station and the global model gradient. The calculation formula is:
[0068] in, These are the current global model parameters. Aggregate weights. The calculation formula is:
[0069] in, This represents the total number of power station nodes participating in federated learning. The system updates the global model based on the aggregate weights, using the following formula:
[0070] These are the updated global model parameters; The aggregated global model is distributed to the local nodes of each power station as the initialization parameters for the next local meta-learning, which alleviates the model drift caused by the non-independent and identically distributed data distribution of different types of energy storage systems, and realizes cross-power station privacy protection and continuous evolution of fault early warning capabilities.
Claims
1. An image recognition and fault early warning system based on energy storage systems, characterized in that, include: Simultaneously acquire time-series video streams and vibration acoustic signals throughout the entire charging and discharging cycle of the energy storage system; The dynamic deformation features and component movement features of the battery cell and module are extracted from the video stream by a time-series motion localization network, and the normal expansion and abnormal bulging deformation, and normal movement and abnormal jamming are distinguished. Modal features of the vibration acoustic signal are extracted, and a cross-modal association dictionary of visual dynamic features and vibration acoustic features is constructed. Based on digital twins, virtual samples of different fault types and degradation levels are generated. Combined with small sample incremental meta-learning algorithm, the cross-modal recognition model is optimized using virtual samples and actual test samples. The system compares measured features with standard features of the digital twin in real time, and outputs fault warning signals, fault levels, and fault evolution trend prediction results.
2. The image recognition and fault early warning system based on energy storage system according to claim 1, characterized in that, Extract the dynamic deformation features and component movement features of the battery cells and modules from the video stream, including: The energy storage cabinet in the time-series video stream is divided into multiple grid nodes, and a spatiotemporal topology graph containing the physical coordinates of the nodes and their adjacency relationships is constructed. The pixel temporal variation sequence of each grid node is used as a node attribute, and the node state is propagated along the topological edges using a graph convolutional network; Manifold learning is introduced to reduce the dimensionality of the high-dimensional spatiotemporal features output by the graph convolutional network, and extract low-dimensional manifold feature vectors that reflect the continuous evolution of cell deformation and component movement in physical space.
3. The image recognition and fault early warning system based on energy storage system according to claim 1, characterized in that, Modal features of the vibration acoustic signal are extracted, including: Phase space reconstruction is performed on the vibration acoustic signal, and the correlation dimension and maximum Lyapunov exponent of the trajectory in the reconstructed phase space are calculated to obtain chaotic features characterizing the nonlinear evolution of early weak faults. The chaotic features are fused with the Mel frequency cepstral coefficients of the acoustic signal in series. Nonnegative matrix factorization is used to decouple the fused features, separating the main modal features reflecting the resonance of the structural body and the residual features reflecting environmental noise interference. The main modal features are used as the final modal features.
4. The image recognition and fault early warning system based on energy storage system according to claim 1, characterized in that, Construct a cross-modal association dictionary of visual dynamic features and vibroacoustic features, including: A dual-stream Transformer architecture is used to perform self-attention encoding on visual dynamic features and vibroacoustic modal features respectively; A cross-modal contrastive learning loss function based on Wasserstein distance is introduced to align the edge distributions and conditional distributions of visual and acoustic flows at the feature distribution level. The aligned multimodal joint features are mapped to a sparse coding space, and the L1 norm regularization optimization problem is solved iteratively to generate a robust cross-modal association dictionary that preserves cross-modal causal relationships.
5. The image recognition and fault early warning system based on energy storage system according to claim 1, characterized in that, Virtual samples with different fault types and degradation levels are generated based on digital twins, including: Construct a physical information neural network with constraints from coupled thermodynamic-electrochemical-structural mechanics partial differential equations; Using the operating boundary conditions and initial state of the energy storage system as input, virtual operating state parameters are output through forward propagation. By coupling the data loss term and the partial differential equation residual term in the loss function of the physical information neural network, the generated virtual samples strictly follow the multi-physics field evolution mechanism of the energy storage system, forming a virtual fault sample library with physical determinism.
6. The image recognition and fault early warning system based on energy storage system according to claim 2, characterized in that, The node states are propagated along the topological edges using a graph convolutional network, including: A dynamic network routing mechanism is introduced to calculate the information transmission gain of each topological edge at the current moment and to globally sort the information transmission gains of all topological edges. Eliminate a predetermined proportion of topological edge state propagation paths that are ranked lower in the sorting, and suppress redundant information interaction between nodes under normal operating conditions. For the retained topological edges, differentiated attention weights are assigned based on the gain value. When a sudden change in the topological structure is detected, long-range skip connections are activated to amplify the weights for state transmission from the abnormally deformed region to the surrounding nodes.
7. The image recognition and fault early warning system based on energy storage system according to claim 3, characterized in that, Before performing phase space reconstruction on the vibration acoustic signal, the following steps are also included: Adaptive wavelet packet decomposition is used to divide the original vibroacoustic signal into multiple scale frequency bands. Using the energy entropy of each sub-band as an individual in the population, a frequency band optimization model based on a genetic algorithm is constructed. The fitness function is to maximize the fault feature distinguishability, and the optimal combination of sub-bands containing the most early fault singular features is selected. Phase space reconstruction is performed only on the selected optimal sub-band combinations to filter out broadband background interference unrelated to the fault.
8. The image recognition and fault early warning system based on energy storage system according to claim 4, characterized in that, Generating a cross-modal association dictionary also includes: The cross-modal association dictionary in the sparse coding space is represented as a high-order tensor, where each dimension of the tensor corresponds to a different fault type, time scale, and modal source. Tensor train decomposition network is used to perform rank-constrained dimensionality reduction on high-order tensors, decomposing the original high-dimensional tensor into a series of low-dimensional core tensors and factor matrices. During the fault warning process, the cross-modal association dictionary is recovered by forward propagation of the low-dimensional core tensor and factor matrix, which reduces the computational complexity and memory usage of multimodal feature joint retrieval.
9. The image recognition and fault early warning system based on energy storage system according to claim 5, characterized in that, Combining a few-sample incremental meta-learning algorithm, the cross-modal recognition model is optimized using virtual and real samples, including: An active learning strategy is introduced into the physically deterministic virtual fault sample library to calculate the prediction entropy and information density of each virtual sample under the current cross-modal recognition model parameters; Boundary support samples are selected based on the joint distribution of prediction entropy and information density; The selected boundary support samples and a small number of measured samples are used to construct a meta-task. By optimizing the parameters in the inner and outer loop training of the meta-task, the cross-modal recognition model is forced to learn how to quickly fine-tune on a small number of measured samples of new energy storage systems.
10. The image recognition and fault early warning system based on energy storage system according to claim 9, characterized in that, This forces cross-modal recognition models to learn how to quickly fine-tune on small measured samples of new energy storage systems, including: A federated learning framework across energy storage power stations is adopted to perform meta-tasks and inner and outer loop parameter updates based on boundary support samples locally at each power station. In the cloud aggregation node, a topology adaptive weighting algorithm is introduced to dynamically allocate aggregation weights based on the cosine similarity between the local model update gradient uploaded by each power station and the global model gradient, thereby alleviating model drift caused by the non-independent and identically distributed data distribution of different types of energy storage systems. The aggregated global model is used as the initialization parameter for the next local meta-learning.