Energy storage system fault prediction method and system based on multi-modal data fusion

Abstract

The invention discloses an energy storage system fault prediction method and system based on multi-modal data fusion, and relates to the technical field of energy storage systems. Comprising the following steps: acquiring multi-modal data, and preprocessing the data to improve the data quality; performing feature extraction on the multi-modal data through a feature extraction model to obtain a feature matrix; performing factorization on the extracted feature matrix, establishing a multi-modal data fusion model, fusing core factor matrixes obtained after decomposition, establishing a prediction model, substituting fused feature vectors for fault prediction, and outputting a prediction result. According to the method, factorization is carried out on the extracted feature matrix through a data decomposition method, the feature decomposition matrix is obtained, core factor matrixes obtained after decomposition are fused through the multi-modal data fusion model, the fused feature vector can further express the mutual relation between multi-modal numerical values, and the fusion efficiency is improved. And the fault prediction accuracy of the energy storage system is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of energy storage systems, and in particular to a method and system for predicting energy storage system faults based on multimodal data fusion. Background Art

[0002] Predicting energy storage system failures in advance allows for appropriate measures to be taken before a failure occurs, such as arranging repairs or replacing components. This prevents sudden failure of the energy storage system and ensures its stable and continuous operation, thereby improving the reliability of the entire power system. Traditional scheduled maintenance can lead to over- or under-maintenance. Fault prediction allows for more targeted maintenance plans based on the system's actual operating conditions and potential failure risks, enabling condition-based maintenance, reducing maintenance costs, and improving efficiency.

[0003] Patent publication number CN118917442A discloses an error change prediction method based on supercapacity energy storage, which includes collecting data information, using a deep learning model for intelligent fault detection, monitoring the operating status of the supercapacity energy storage system, identifying and marking abnormal data, and correcting and eliminating the marked abnormal data; using a wavelet transform method to perform multi-scale analysis on the corrected data, extracting multi-level features of error changes, and further extracting nonlinear features in error changes through nonlinear feature extraction technology; constructing a hybrid prediction model to predict error changes, deploying a lightweight prediction model on the edge device, performing real-time data processing and prediction, and uploading the results to the cloud. The method of the present invention can not only effectively reduce the limitations and instability of a single model, but also optimize the prediction results through an integrated strategy, thereby improving the robustness and accuracy of the model.

[0004] In energy storage system fault prediction, it is necessary to collect a variety of data, such as voltage, current, temperature, and maintenance records. By analyzing these data, the possible occurrence time of the fault can be predicted. The above method can pre-process the collected multimodal data and use multiple different prediction methods to make separate predictions during the prediction process. Finally, the prediction results of multiple prediction methods are combined. However, these multimodal data are processed separately during the processing process and brought into the prediction model. The mutual correlation between the different modal values is not combined to predict the fault, and the prediction results are not accurate enough. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for energy storage system fault prediction based on multimodal data fusion to solve the problems raised in the above background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for energy storage system fault prediction based on multimodal data fusion, comprising:

[0007] Acquire historical fault data and multimodal data, the multimodal data including text data, numerical data, and image data, and preprocess the data to improve data quality;

[0008] Establish a feature extraction model, and use the feature extraction model to extract features from text data, numerical data, and image data to obtain a feature matrix;

[0009] The extracted feature matrix is factored through data decomposition methods to obtain a feature decomposition matrix. The feature decomposition matrix includes a core factor matrix or several other factor matrices. The high-dimensional multimodal data is decomposed into the sum of multiple low-dimensional feature decomposition matrices to extract the potential structure and core features of the data, reduce the data dimension, and reduce the difficulty of processing and analysis.

[0010] A multimodal data fusion model is established, and the core factor matrix obtained after decomposition is fused through the multimodal data fusion model to obtain a fused feature vector. The fused feature vector can further represent the relationship between multimodal values and improve the accuracy of energy storage system fault prediction.

[0011] A prediction model is established based on historical fault data, the fused feature vector is brought into the prediction model for fault prediction, and the prediction results are output.

[0012] Preferably, the establishing of the feature extraction model includes:

[0013] Establish a numerical data extraction model: combine the numerical data into a numerical data matrix with a height of 1 according to the acquisition time and feature items, and perform normalization. The processing formula is as follows:

[0014]

[0015] in is the jth numerical feature of the i-th sample, which is represented by the value of the position with coordinates (i, j) in the feature matrix. and are the minimum and maximum values of the j-th numerical feature, respectively. is the normalized value of the jth numerical feature of the i-th sample, which is represented by the value of the position with coordinates (i, j) in the feature matrix;

[0016] Establish an image data extraction model: convert the image data into a single-channel matrix to obtain the image data matrix, and perform a convolution operation on the image data matrix. The processing formula is as follows:

[0017]

[0018] in is the value after convolution at the position with the horizontal coordinate (m, n) on the image, and is represented as the value at the position with the coordinate (m, n) in the feature matrix. K uv is the value of the convolution kernel K at the position with the horizontal coordinate u and the vertical coordinate v, where U and V are the length and width of the convolution kernel respectively;

[0019] Establish a text data extraction model: Create a list of terms, perform text recognition on multiple sets of text data, record the number of times each term appears, and form each set of text data into a one-dimensional feature matrix. The processing formula is as follows:

[0020]

[0021] in is the corresponding calculated value of the term k in the term list, which is represented by the value of the kth position in the feature matrix, c k is the number of times the term k appears in the text data, N is the total number of terms, and in the feature matrix it is represented by the total length of the matrix;

[0022] Then, the one-dimensional feature matrices obtained from multiple text data are superimposed to obtain a two-dimensional feature matrix.

[0023] Preferably, the establishment of the image data extraction model is specifically as follows:

[0024] Get multiple image data, convert each image data into a single-channel matrix, obtain the image data matrix, and perform convolution operation on the image data matrix. The processing formula is as follows:

[0025]

[0026] in is the value after convolution at the position with the horizontal coordinate (m, n) on the image, and is represented as the value at the position with the coordinate (m, n) in the feature matrix. K uv is the value of the convolution kernel K at the position with the horizontal coordinate u and the vertical coordinate v, where U and V are the length and width of the convolution kernel respectively;

[0027] Then perform matrix superposition calculation on the feature matrices obtained from multiple images to obtain a three-dimensional feature matrix. The formula is:

[0028]

[0029] in is a three-dimensional matrix f i At the element with coordinates (m, n, j), is the j-th two-dimensional matrix The element at coordinates (m, n).

[0030] Preferably, the data decomposition method includes:

[0031] According to the dimension of the feature matrix obtained from multimodal data, dimensional decomposition is performed, and the decomposition formula is:

[0032]

[0033] where σ i feature matrix, is the weight of the r-th core factor matrix, is the core factor matrix, R is the number of core factor matrices, is the vector outer product operator symbol, Indicates continuous The number of operations, W is the dimension of the feature matrix, is the other factor matrix.

[0034] Preferably, the multimodal data fusion model is specifically:

[0035]

[0036] where ω n 、ω i 、ω t are the fusion weights of numerical, image, and text data respectively, is the dth value in the core factor matrix of numerical data, is the weight of the core factor matrix of numerical data, is the dth value in the core factor matrix of the image data, is the weight of the core factor matrix of the image data, is the dth value in the core factor matrix of text data, is the weight of the core factor matrix of text data, X Yd is the dth value of the fused feature vector.

[0037] Preferably, the historical fault data is used to establish a prediction model, specifically:

[0038] Establish a preliminary prediction formula, specifically:

[0039]

[0040] Where β0 is the bias term, d is the number of values in the fused feature vector, β j is the coefficient of the jth feature in the fusion feature vector, X Yj is the value of the jth feature in the fused feature vector, is the predicted probability;

[0041] Collect historical data and train the fault prediction model through the model training formula to obtain β0 and βj The specific value of .

[0042] Preferably, the model training formula is specifically:

[0043]

[0044] Where L is the loss constant, is the predicted probability of the fault prediction model for sample i, Y i is the true fault label of sample i, M is the number of samples;

[0045] Continuously adjust β0 and β through optimization algorithm j The value of is set to minimize the loss constant L, thereby obtaining the optimal prediction model for fault prediction of the energy storage system.

[0046] Preferably, the prediction result is compared with a preset threshold to determine whether the failure probability exceeds the limit. If the prediction exceeds the threshold, the system issues a warning message and feeds back the alarm information to relevant personnel to ensure that the problem is handled in a timely manner and to ensure the safe and stable operation of the system.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The extracted feature matrix is factored through the data decomposition method to obtain the feature decomposition matrix, and then the core factor matrix obtained after decomposition is fused through the multimodal data fusion model, and the high-dimensional multimodal data is decomposed into the sum of multiple low-dimensional feature decomposition matrices to extract the potential structure and core features of the data and reduce the data dimension. The fused feature vector after fusion can further represent the relationship between multimodal values and improve the accuracy of energy storage system failure prediction.

[0049] At the same time, different feature extraction models are used to extract features from text data, numerical data, and image data respectively, and the multimodal data is converted into a feature matrix, and its expression form is unified for easy processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of the process of the fault prediction method of the present invention;

[0051] Figure 2 Schematic diagram of the process of the feature extraction model of the present invention;

[0052] Figure 3 Schematic diagram of the data decomposition method of the present invention;

[0053] Figure 4 Schematic diagram of the process of the multimodal data fusion model of the present invention;

[0054] Figure 5 Schematic diagram of the process of establishing a prediction model of the present invention. DETAILED DESCRIPTION

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0056] In this application, for ease of understanding, the method steps used do not need to be executed in the order of the steps in this embodiment during actual operation. In other embodiments, these steps may be performed simultaneously or in a different order.

[0057] Example 1:

[0058] Fault prediction in energy storage systems requires the collection of a variety of data, such as voltage, current, temperature, and maintenance records. By analyzing this data, the possible occurrence time of the fault can be predicted. Combined with the intercorrelations between different modal values, fault prediction can be performed to improve prediction accuracy.

[0059] like Figure 1-Figure 5 As shown, the present invention provides a technical solution: a method for energy storage system fault prediction based on multimodal data fusion, comprising:

[0060] Acquire historical fault data and multimodal data, including text data, numerical data, and image data, and preprocess the data to improve data quality;

[0061] Establish a feature extraction model, and use the feature extraction model to extract features from text data, numerical data, and image data to obtain a feature matrix;

[0062] It should be noted that multimodal data can be obtained through various sensors and technicians' records. Numerical data includes data expressed in numerical values such as voltage, current, temperature, and battery internal resistance. Text data includes maintenance record sheets and external interference factor record sheets. Image data includes pictures of batteries and equipment and infrared temperature distribution maps. These can be obtained through existing technologies and will not be elaborated here.

[0063] The extracted feature matrix is factored through data decomposition methods to obtain a feature decomposition matrix. The feature decomposition matrix includes a core factor matrix or several other factor matrices. The high-dimensional multimodal data is decomposed into the sum of multiple low-dimensional feature decomposition matrices to extract the potential structure and core features of the data, reduce the data dimension, and reduce the difficulty of processing and analysis.

[0064] A multimodal data fusion model is established, and the core factor matrix obtained after decomposition is fused through the multimodal data fusion model to obtain a fused feature vector. The fused feature vector can further represent the relationship between multimodal values and improve the accuracy of energy storage system fault prediction.

[0065] Establish a prediction model based on historical fault data, bring the fused feature vector into the prediction model to perform fault prediction, and output the prediction results;

[0066] It should be noted that the prediction results are compared with the preset threshold to determine whether the failure probability exceeds the limit. If the prediction exceeds the threshold, the system will issue a warning message and feed the alarm information back to the relevant personnel to ensure that the problem is handled in a timely manner and the safe and stable operation of the system is guaranteed.

[0067] like Figure 2 As shown, a feature extraction model is established, including:

[0068] Establish a numerical data extraction model: combine the numerical data into a numerical data matrix with a height of 1 according to the acquisition time and feature items, and perform normalization. The processing formula is as follows:

[0069]

[0070] in is the jth numerical feature of the i-th sample, which is represented by the value of the position with coordinates (i, j) in the feature matrix. and are the minimum and maximum values of the j-th numerical feature, respectively. is the normalized value of the jth numerical feature of the i-th sample, which is represented by the value of the position with coordinates (i, j) in the feature matrix;

[0071] Establish an image data extraction model: convert the image data into a single-channel matrix to obtain the image data matrix, and perform a convolution operation on the image data matrix. The processing formula is as follows:

[0072]

[0073] in is the value after convolution at the position with the horizontal coordinate (m, n) on the image, and is represented as the value at the position with the coordinate (m, n) in the feature matrix. K uv is the value of the convolution kernel K at the position with the horizontal coordinate u and the vertical coordinate v, where U and V are the length and width of the convolution kernel respectively;

[0074] Establish a text data extraction model: Create a list of terms, perform text recognition on multiple sets of text data, record the number of times each term appears, and form each set of text data into a one-dimensional feature matrix. The processing formula is as follows:

[0075]

[0076] in is the corresponding calculated value of the term k in the term list, which is represented by the value of the kth position in the feature matrix, c k is the number of times the term k appears in the text data, N is the total number of terms, and in the feature matrix it is represented by the total length of the matrix;

[0077] Then, the one-dimensional feature matrices obtained from multiple text data are superimposed to obtain a two-dimensional feature matrix.

[0078] It should be noted that for ease of understanding, the simulation data is set as follows:

[0079] For numerical data, the voltage data of 5 samples at 4 different time points were monitored, as shown in Table 1:

[0080] Table 1: Monitoring sample table

[0081]

[0082]

[0083] For image data, set the convolution kernel The appearance of the energy storage device was photographed, and the original image data obtained was shown in Table 2 (different positions represent different pixels):

[0084] Table 2: Image pixel map

[0085] 2 3 4 5 6 7 8 9 10

[0086] For text data, we collected 5 equipment maintenance records (corresponding to 5 samples). After counting, there are 8 entries in the entry list. The number of occurrences of each word is shown in Table 3 (the data is the number of occurrences of the entry):

[0087] Table 3: Text entry table

[0088] Entry 1 Entry 2 Entry 3 Entry 4 Entry 5 Entry 6 Entry 7 Entry 8 Sample 1 1 0 2 0 1 0 0 0 Sample 2 0 1 0 0 2 0 1 0 Sample 3 1 1 1 0 1 0 0 0 Sample 4 2 1 0 0 0 0 0 0 Sample 5 0 0 0 2 0 0 0 0

[0089] Perform feature extraction on numerical data:

[0090] Taking the first feature of the first sample as an example, According to the formula Calculation can be obtained

[0091] The feature matrix obtained by extracting the entire numerical data is shown in Table 4:

[0092] Table 4: Numerical feature matrix

[0093] 0.375 0.545 0.78 0.75 0 0 0 0 1 1 1 1 0.5 0.273 0.385 0.333 0.875 0.636 0.769 1

[0094] Extract features from image data. When m=1, n=1, the formula Calculation can be obtained ×1+5×(-1)+6×(-1))=-6.

[0095] The feature matrix obtained by extracting the entire image data (filling the edges of the image with the median, and the average or other filling methods can be used in practice) is shown in Table 5:

[0096] Table 5: Image data feature matrix

[0097] -6 -6 -3 -6 -6 -3 5 7 -4

[0098] Extract features from text data. Taking the first record as an example, according to the formula Calculation can be obtained

[0099] The feature matrix obtained by feature extraction of the entire text data is shown in Table 6:

[0100] Table 6: Text data feature matrix

[0101] 0.25 0 0.5 0 0.25 0 0 0 0 0.25 0 0 0.5 0 0.25 0 0.25 0.25 0.25 0 0.25 0 0 0 0.66 0.33 0 0 0 0 0 0 0 0 0 1 0 0 0 0

[0102] Different feature extraction models are used to extract features from numerical data, text data, and images respectively, and data of different modalities are converted into feature matrices to unify the data format, making it easier to subsequently integrate multimodal data.

[0103] refer to Figure 3 As shown, the data decomposition methods include:

[0104] According to the dimension of the feature matrix obtained from multimodal data, dimensional decomposition is performed, and the decomposition formula is:

[0105]

[0106] where σ i feature matrix, is the weight of the r-th core factor matrix, is the core factor matrix, R is the number of core factor matrices, is the vector outer product operator symbol, Indicates continuous The number of operations, W is the dimension of the feature matrix, is the other factor matrix.

[0107] It should be noted that the above simulation data is used for calculation. Taking the numerical feature matrix as an example, the matrix dimension is 2, so W = 2. For the convenience of calculation, R = 2 is set. The calculation based on the alternating least squares method (ALS) can be obtained (here is simplified data):

[0108] It should be noted that due to and For other factor matrices, they do not participate in subsequent calculations and are not shown in the embodiment. Other factor matrices of subsequent text data and image data are also omitted. Alternating least squares (ALS) is an existing calculation method. In this embodiment, the core factor matrix and other factor matrices can be fixed in turn, and repeated iterations can be performed until the error between two iterations is less than a threshold, or the preset maximum number of iterations is reached. Other calculation methods can also be used for calculation, which will not be elaborated here.

[0109] Similarly, the feature matrix of the image data can be calculated (here is hypothetical data):

[0110]

[0111] Similarly, the feature matrix of text data can be calculated (here is hypothetical data):

[0112]

[0113] The feature matrices of different dimensions are converted into a single-dimensional core factor matrix that best reflects the characteristics of the original data, and the high-dimensional multimodal data is decomposed into the sum of multiple low-dimensional feature decomposition matrices to extract the potential structure and core features of the data, reduce the data dimension, and reduce the difficulty of processing and analysis.

[0114] refer to Figure 4 As shown in Figure 2, the multimodal data fusion model is specifically:

[0115]

[0116] where ω n 、ω i 、ω t are the fusion weights of numerical, image, and text data respectively, is the dth value in the core factor matrix of numerical data, is the weight of the core factor matrix of numerical data, is the dth value in the core factor matrix of the image data, is the weight of the core factor matrix of the image data, is the dth value in the core factor matrix of text data, is the weight of the core factor matrix of text data, X Yd is the dth value of the fused feature vector.

[0117] It should be noted that the above simulation data is used for calculation, assuming that the weight of the numerical data ω n =0.4, weight of image data ω i =0.3, weight of text data ω t = 0.3, taking the first value of the fused feature vector as an example to calculate X Y1 =0.4×(0.7×0.5+0.5×0.2)+0.3×(0.6×0.6+0.4×0.3)+0.3×(0.8×0.5+0.3×0.2)=0.18+0.144+0.138=0.462.

[0118] By calculating the values of each core factor matrix in turn (the number of core factor matrices for each modal data is different, and the number of elements in each core factor matrix may be different, and they are filled with 0 values during calculation), a complete fused eigenvector can be obtained. The fused eigenvector can further represent the relationship between multi-modal values and improve the accuracy of energy storage system fault prediction.

[0119] refer to Figure 5 As shown in the figure, a prediction model is established based on historical fault data, specifically:

[0120] Establish a preliminary prediction formula, specifically:

[0121]

[0122] Where β0 is the bias term, d is the number of values in the fused feature vector, β j is the coefficient of the jth feature in the fusion feature vector, X Yj is the value of the jth feature in the fused feature vector, is the predicted probability;

[0123] Collect historical data and train the fault prediction model through the model training formula to obtain β0 and β j The specific value of .

[0124] It should be noted that, in order to facilitate the calculation, the above simulation data is used for calculation, assuming that the fusion feature vector has only one element, that is, d = 1, β0 = 0.1, β j =0.2, through the above calculation formula Calculation can be obtained

[0125] The model training formula is as follows:

[0126]

[0127] Where L is the loss constant, is the predicted probability of the fault prediction model for sample i, Y i is the true fault label of sample i, M is the number of samples;

[0128] Continuously adjust β0 and β through optimization algorithm j The value of is set to minimize the loss constant L, thereby obtaining the optimal prediction model for fault prediction of the energy storage system.

[0129] It should be noted that, assuming the number of samples M = 5, the true fault label is (0, 1, 0, 1, 0), where 0 represents no fault and 1 represents a fault. The accuracy of the prediction is determined by calculating L. The goal of model training is to adjust β0 and β j The value of minimizes the loss constant L, thereby improving the prediction performance of the model. The calculation process can be implemented using existing algorithms such as gradient descent, Newton's method, or BFGS algorithm, which will not be described in detail here. The smaller the value of the loss constant L, the more accurate the prediction result of the prediction model.

[0130] Example 2:

[0131] When acquiring image data, images of different positions may be acquired, such as images of wiring joints and battery edges, resulting in multiple groups of image data. In Example 1, using one group of images for calculation may result in inaccurate prediction results. The difference between this embodiment and Example 1 is that multiple groups of image data can be used for feature extraction to improve the accuracy of the prediction results.

[0132] Building an image data extraction model also includes:

[0133] Get multiple image data, convert each image data into a single-channel matrix, obtain the image data matrix, and perform convolution operation on the image data matrix. The processing formula is as follows:

[0134]

[0135] in is the value after convolution at the position with the horizontal coordinate (m, n) on the image, and is represented as the value at the position with the coordinate (m, n) in the feature matrix. K uv is the value of the convolution kernel K at the position with the horizontal coordinate u and the vertical coordinate v, where U and V are the length and width of the convolution kernel respectively;

[0136] Then perform matrix superposition calculation on the feature matrices obtained from multiple images to obtain a three-dimensional feature matrix. The formula is:

[0137]

[0138] in is a three-dimensional matrix f i At the element with coordinates (m, n, j), is the j-th two-dimensional matrix The element at coordinates (m, n).

[0139] It should be noted that when there are multiple sets of image data, each image data can be convolved to obtain multiple sets of two-dimensional image data feature matrices as shown in Table 5. By superimposing the multiple sets of two-dimensional image data feature matrices in the height direction, a three-dimensional image data feature matrix can be obtained. When decomposing by the data decomposition method, the dimension of the feature matrix can be W = 3. The feature matrices of numerical data and text data can be expanded in a two-dimensional form, and the computational complexity is low. The more samples of image data, the more complex the three-dimensional form of its feature evidence, but the superimposed feature matrix will respond to the actual situation more carefully and realistically, which can further improve the accuracy of fault prediction.

[0140] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is limited by the accompanying embodiments and their equivalents.

Claims

1. A method for energy storage system fault prediction based on multimodal data fusion, characterized by: include: Acquire historical fault data and multimodal data, the multimodal data including text data, numerical data, and image data, and preprocess the data to improve data quality; Establish a feature extraction model, and use the feature extraction model to extract features from text data, numerical data, and image data to obtain a feature matrix; The extracted feature matrix is factored through data decomposition methods to obtain a feature decomposition matrix. The feature decomposition matrix includes a core factor matrix or several other factor matrices. The high-dimensional multimodal data is decomposed into the sum of multiple low-dimensional feature decomposition matrices to extract the potential structure and core features of the data, reduce the data dimension, and reduce the difficulty of processing and analysis. A multimodal data fusion model is established, and the core factor matrix obtained after decomposition is fused through the multimodal data fusion model to obtain a fused feature vector. The fused feature vector can further represent the relationship between multimodal values and improve the accuracy of energy storage system fault prediction. A prediction model is established based on historical fault data, the fused feature vector is brought into the prediction model for fault prediction, and the prediction results are output.

2. The method for energy storage system fault prediction based on multimodal data fusion according to claim 1, characterized in that: The establishing of the feature extraction model comprises: Establish a numerical data extraction model: combine the numerical data into a numerical data matrix with a height of 1 according to the acquisition time and feature items, and perform normalization. The processing formula is as follows: in is the jth numerical feature of the i-th sample, which is represented by the value of the position with coordinates (i, j) in the feature matrix. and are the minimum and maximum values of the j-th numerical feature, respectively. is the normalized value of the jth numerical feature of the i-th sample, which is represented by the value of the position with coordinates (i, j) in the feature matrix; Establish an image data extraction model: convert the image data into a single-channel matrix to obtain the image data matrix, and perform a convolution operation on the image data matrix. The processing formula is as follows: in is the value after convolution at the position with the horizontal coordinate (m, n) on the image, and is represented as the value at the position with the coordinate (m, n) in the feature matrix. K uv is the value of the convolution kernel K at the position with the horizontal coordinate u and the vertical coordinate v, where U and V are the length and width of the convolution kernel respectively; Establish a text data extraction model: Create a list of terms, perform text recognition on multiple sets of text data, record the number of times each term appears, and form each set of text data into a one-dimensional feature matrix. The processing formula is as follows: in is the corresponding calculated value of the term k in the term list, which is represented by the value of the kth position in the feature matrix, c k is the number of times the term k appears in the text data, N is the total number of terms, and in the feature matrix it is represented by the total length of the matrix; Then, the one-dimensional feature matrices obtained from multiple text data are superimposed to obtain a two-dimensional feature matrix.

3. The method for energy storage system fault prediction based on multimodal data fusion according to claim 2, characterized in that: The establishment of the image data extraction model is specifically as follows: Get multiple image data, convert each image data into a single-channel matrix, obtain the image data matrix, and perform convolution operation on the image data matrix. The processing formula is as follows: in is the value after convolution at the position with the horizontal coordinate (m, n) on the image, and is represented as the value at the position with the coordinate (m, n) in the feature matrix. K uv is the value of the convolution kernel K at the position with the horizontal coordinate u and the vertical coordinate v, where U and V are the length and width of the convolution kernel respectively; Then perform matrix superposition calculation on the feature matrices obtained from multiple images to obtain a three-dimensional feature matrix. The formula is: in is a three-dimensional matrix f i At the element with coordinates (m, n, j), is the j-th two-dimensional matrix The element at coordinates (m, n).

4. The method for energy storage system fault prediction based on multimodal data fusion according to claim 1, characterized in that: The data decomposition method comprises: According to the dimension of the feature matrix obtained from multimodal data, dimensional decomposition is performed, and the decomposition formula is: where σ i feature matrix, is the weight of the r-th core factor matrix, is the core factor matrix, R is the number of core factor matrices, is the vector outer product operator symbol, Indicates continuous The number of operations, W is the dimension of the feature matrix, is the other factor matrix.

5. The method for energy storage system fault prediction based on multimodal data fusion according to claim 1, characterized in that: The multimodal data fusion model is specifically: where ω n 、ω i 、ω t are the fusion weights of numerical, image, and text data respectively, is the dth value in the core factor matrix of numerical data, is the weight of the core factor matrix of numerical data, is the dth value in the core factor matrix of the image data, is the weight of the core factor matrix of the image data, is the dth value in the core factor matrix of text data, is the weight of the core factor matrix of text data, X Yd is the dth value of the fused feature vector.

6. The method for energy storage system fault prediction based on multimodal data fusion according to claim 1, characterized in that: The prediction model is established based on historical fault data, specifically: Establish a preliminary prediction formula, specifically: Where β0 is the bias term, d is the number of values in the fused feature vector, β j is the coefficient of the jth feature in the fusion feature vector, X Yj is the value of the jth feature in the fused feature vector, is the predicted probability; Collect historical data and train the fault prediction model through the model training formula to obtain β0 and β j The specific value of .

7. The method for energy storage system fault prediction based on multimodal data fusion according to claim 5, characterized in that: The model training formula is specifically: Where L is the loss constant, is the predicted probability of the fault prediction model for sample i, Y i is the true fault label of sample i, M is the number of samples; Continuously adjust β0 and β through optimization algorithm j The value of is set to minimize the loss constant L, thereby obtaining the optimal prediction model for fault prediction of the energy storage system.

8. The method for energy storage system fault prediction based on multimodal data fusion according to claim 1, characterized in that: The prediction result is compared with the preset threshold to determine whether the failure probability exceeds the limit. If the prediction exceeds the threshold, the system will issue a warning message and feedback the alarm information to relevant personnel to ensure that the problem is handled in a timely manner and ensure the safe and stable operation of the system.

9. A fault prediction system for energy storage systems based on multimodal data fusion, characterized by: include: Data collection module: used to obtain historical fault data and multimodal data, the multimodal data including text data, numerical data and image data; Data sorting module: used to pre-process data, improve data quality, establish a feature extraction model, and use the feature extraction model to extract features from text data, numerical data, and image data to obtain feature matrices; Data processing module: used to factorize the extracted feature matrix to obtain the feature decomposition matrix, which includes a core factor matrix or several other factor matrices. It decomposes the high-dimensional multimodal data into the sum of multiple low-dimensional feature decomposition matrices, extracts the potential structure and core features of the data, reduces the data dimension, and reduces the difficulty of processing and analysis; establishes a multimodal data fusion model, and fuses the core factor matrix obtained after decomposition through the multimodal data fusion model to obtain a fused feature vector. The fused feature vector can further represent the relationship between multimodal values and improve the accuracy of energy storage system fault prediction; Data output module: used to bring the fused feature vector into the prediction model for fault prediction and output the prediction results.

Images