Emergency power supply state monitoring method and system based on multi-modal data

Abstract

The invention discloses an emergency power supply state monitoring method and system based on multi-modal data, and relates to the technical field of power supply monitoring, and the method comprises the steps: obtaining the power supply attribute data of an emergency power supply; based on the sensor group, collecting multi-channel time sequence sampling data of the emergency power supply corresponding to a preset sampling time period in real time; performing data preprocessing on the multi-channel time sequence sampling data to obtain corresponding online monitoring time sequence data; extracting a multi-modal fusion feature group corresponding to the online monitoring time sequence data; and inputting the power supply attribute data and the multi-modal fusion feature group into a power supply state identification model to determine the operation health state of the emergency power supply, the operation health state including any one of the following state types: a normal state, an early warning state, a limited operation state and a fault state. Therefore, technical means such as multi-modal parameter acquisition, feature fusion analysis and model reasoning are comprehensively adopted, and real-time and accurate monitoring of the state of the emergency power supply is realized.

Description

Technical Field

[0001] The present application relates to the technical field of power supply monitoring based on data signal analysis, and in particular to a method and system for monitoring emergency power supply status based on multimodal data. Background Art

[0002] In modern society, emergency power supply systems are an important guarantee for the normal operation of key scenarios (such as hospitals, data centers, airports, financial institutions, etc.). When the main power supply is interrupted, the emergency power supply or UPS (Uninterruptible Power Supply) must be started immediately and maintain a stable power supply. However, the UPS emergency power supply may face various complex situations during operation, such as aging of internal components and external environmental interference. These factors may affect its performance or even cause failures, which in turn have a serious impact on important systems that rely on it for power supply. Therefore, accurate and timely monitoring of the status of the emergency power supply has become a key link in ensuring power supply safety.

[0003] Traditional emergency power supply status monitoring methods often rely on simple electrical parameter measurements, such as basic value monitoring of voltage and current. This single data source monitoring method cannot fully reflect the complex operating status of the power supply. In addition, traditional monitoring systems are usually based on threshold value judgments or linear models. Such methods assume that the relationship between various parameters is linear. However, in actual operation, the status of emergency power supplies is affected by a variety of nonlinear factors, such as sudden load changes, temperature changes, or degradation of internal battery chemical reactions. Simple linear models show low prediction accuracy in these cases and cannot effectively capture the dynamic changes in the status of emergency power supplies.

[0004] In response to the above problems, the industry has not yet proposed a better technical solution. Summary of the invention

[0005] The present application provides an emergency power supply status monitoring method, system, storage medium, computer program product and electronic device based on multimodal data, which are used to at least solve the problems of inaccurate power supply status capture and untimely early warning response caused by the single dimension of power supply monitoring data in the current related technology.

[0006] In a first aspect, an embodiment of the present application provides an emergency power supply status monitoring method based on multimodal data, including: obtaining power supply attribute data of the emergency power supply; the power supply attribute data includes power supply capacity, power supply rated power, battery internal resistance, historical charge and discharge cycle data and historical abnormality records; based on a sensor group, real-time acquisition of multi-channel timing sampling data of the emergency power supply corresponding to a preset sampling time period; the sensor group includes a current sensor, a voltage sensor, an acoustic wave sensor and a photoelectric sensor; the multi-channel timing sampling data includes current timing parameters, voltage timing parameters, acoustic wave timing parameters and light sensing timing parameters; The time series sampling data is preprocessed to obtain corresponding online monitoring time series data; a multimodal fusion feature group corresponding to the online monitoring time series data is extracted; the multimodal fusion feature group includes time domain features, frequency domain features, time-frequency domain features and space domain features; the power attribute data and the multimodal fusion feature group are input into a power state recognition model to determine the operating health state of the emergency power supply; the operating health state includes any one of the following state types: normal state, warning state, restricted operating state and fault state; the power state recognition model adopts a time series classification model based on an attention mechanism.

[0007] In a second aspect, an embodiment of the present application provides an emergency power supply status monitoring system based on multimodal data, including: an attribute data acquisition unit, used to acquire power supply attribute data of the emergency power supply; the power supply attribute data includes power supply capacity, power supply rated power, battery internal resistance, historical charge and discharge cycle data and historical abnormality records; a sensor data acquisition unit, used to collect multi-channel timing sampling data of the emergency power supply corresponding to a preset sampling time period in real time based on a sensor group; the sensor group includes a current sensor, a voltage sensor, an acoustic wave sensor and a photoelectric sensor; the multi-channel timing sampling data includes current timing parameters, voltage timing parameters, acoustic wave timing parameters and light sensing timing parameters; a data preprocessing unit, used to The multi-channel time series sampling data is preprocessed to obtain corresponding online monitoring time series data; a multimodal feature extraction unit is used to extract a multimodal fusion feature group corresponding to the online monitoring time series data; the multimodal fusion feature group includes time domain features, frequency domain features, time-frequency domain features and space domain features; an operation status identification unit is used to input the power supply attribute data and the multimodal fusion feature group into a power supply status identification model to determine the operation health status of the emergency power supply; the operation health status includes any one of the following state types: normal state, warning state, restricted operation state and fault state; the power supply status identification model adopts a time series classification model based on an attention mechanism.

[0008] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the emergency power supply status monitoring method based on multimodal data of any embodiment of the present application.

[0009] In a fourth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of the emergency power supply status monitoring method based on multimodal data of any embodiment of the present application are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the emergency power supply status monitoring method based on multimodal data of any embodiment of the present application.

[0011] The emergency power supply status monitoring method and system based on multimodal data provided by the present application can produce at least the following technical effects: (1) It integrates the time series data collected by multiple sensors such as current, voltage, sound waves, and light, and combines the historical charge and discharge records, internal resistance changes, historical faults and other attribute information of the power supply to achieve comprehensive monitoring of the operating status of the emergency power supply from multiple dimensions such as electrical characteristics, physical signals, and environmental parameters. Compared with single voltage or current monitoring, this multimodal monitoring method can capture more subtle and complex changes. For example, through sound waves and photoelectric sensors, possible signs of degradation or environmental interference inside the battery can be detected, which can more accurately reflect the health status of the emergency power supply.

[0012] (2) By extracting the time domain, frequency domain, time-frequency domain, and space domain features of multimodal data, it is possible to capture the nonlinear changes in the emergency power supply state. By using multimodal feature fusion analysis, the complex correlation between different physical quantities under various power supply states can be discovered, thereby detecting abnormal trends in time before equipment failure and avoiding continuous deterioration of the state. This can effectively improve the sensitivity and accuracy of abnormal state detection.

[0013] (3) By adopting an online monitoring method based on multimodal data and integrating historical data with the currently monitored multimodal real-time data into the state recognition model, the degradation trend or failure risk of the emergency power supply can be effectively identified. The operating status of the equipment (normal, warning, restricted operation and failure) can be accurately judged according to different characteristic patterns. It can support the dynamic implementation of alarm strategies for different states under the influence of nonlinear factors such as environmental disturbances, thereby improving the timeliness and accuracy of the warning.

[0014] Through this technical solution, multimodal parameter collection, feature fusion analysis, model reasoning and other technical means are comprehensively adopted to achieve real-time and accurate monitoring of the emergency power supply status. It can not only capture complex nonlinear changes, but also implement dynamic alarms according to the operating status, providing a solid guarantee for the safe operation of the emergency power supply and ensuring that key systems can obtain reliable power support when the city power is interrupted. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 A flowchart of an example of a method for monitoring emergency power supply status based on multimodal data according to an embodiment of the present application is shown; Figure 2 An operation flow chart of an example of extracting a multimodal fusion feature group according to an embodiment of the present application is shown; Figure 3 A schematic diagram showing a structural connection of an example of a power state recognition model according to an embodiment of the present application is shown; Figure 4 A schematic diagram showing a structural connection of an example of a hybrid network layer according to an embodiment of the present application is shown; Figure 5 A structural block diagram of an example of an emergency power supply status monitoring system based on multimodal data according to an embodiment of the present application is shown; Figure 6 It is a schematic structural diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0019] Figure 1A flowchart of an example of an emergency power supply status monitoring method based on multimodal data according to an embodiment of the present application is shown.

[0020] Regarding the execution subject of the method of the embodiment of the present application, it can be any controller or processor with computing or processing capabilities. Specifically, it can be implemented by an emergency power supply status monitoring server or terminal, for example, through the combination of multimodal data acquisition and feature fusion, real-time monitoring and model recognition. Compared with traditional monitoring methods, it can not only capture complex nonlinear changes, but also realize dynamic alarm adjustment based on the analyzed different operating states, providing a solid guarantee for the safe operation of the emergency power supply, thereby improving the uninterrupted power supply guarantee level of key systems such as industrial production, military nuclear power facilities and communication networks.

[0021] In some examples, it may be integrated and configured in an electronic device or terminal by means of software, hardware, or a combination of software and hardware, and the type of the terminal or electronic device may be diverse, such as a mobile phone, a tablet computer, or a desktop computer, etc.

[0022] like Figure 1 As shown, in step S110, power source attribute data of the emergency power source is obtained.

[0023] Here, the power attribute data includes power capacity, power rated power, battery internal resistance, historical charge and discharge cycle data, and historical abnormal records. In some embodiments, the above power attribute data is obtained through a battery management system (BMS).

[0024] Regarding the selection instructions for the above parameters, the power supply capacity is used to reflect the power storage capacity of the emergency power supply and determines its maximum power supply time. The power supply rated power is used to reflect the maximum power that the power supply can output during normal operation. The battery internal resistance is used to reflect the degree of aging of the internal materials of the battery and is an important indicator of battery health. The historical charge and discharge cycle data includes the number of charge and discharge cycles, the depth and duration of each discharge, as well as key information such as charging voltage and current. It is used to analyze the life of the UPS emergency power supply and evaluate long-term operating performance. The historical abnormality record includes the frequency of past operating abnormalities of the UPS emergency power supply and the corresponding parameter records, which is conducive to the model learning the abnormal operating mode of the emergency power supply and improving the accuracy of status monitoring.

[0025] In step S120, based on the sensor group, multi-channel time series sampling data of the emergency power supply corresponding to the preset sampling time period is collected in real time.

[0026] Here, the sensor group includes current sensors, voltage sensors, acoustic wave sensors and photoelectric sensors, and the corresponding multi-channel timing sampling data includes current timing parameters, voltage timing parameters, acoustic wave timing parameters and light sensing timing parameters. The current sensor is used to monitor the current changes during the operation of the equipment and capture current fluctuations, such as sudden load changes. The voltage sensor is used to monitor the output voltage of the power supply and its fluctuations in real time to identify whether there is voltage instability or voltage abnormality. The acoustic wave sensor is used to detect the physical state inside the battery, such as gas release or mechanical vibration signals of battery swelling. The optical signal changes caused by battery abnormalities are monitored by photoelectric sensors, such as spark signals caused by poor line contact, deformation signals of battery swelling, and so on.

[0027] In some examples of the embodiments of the present application, the acoustic wave sensor is installed on the inner surface of the shell of the emergency power supply. As the internal materials of the battery deteriorate, some slight sound or vibration signals will be generated. Installing the sensor on the inner surface of the shell enables the sensor to capture small abnormalities in the internal structure of the battery at close range, such as electrolyte stratification, battery plate deformation or battery swelling, etc., thereby improving the accuracy of detection and the ability to monitor the aging trend of the battery. The photoelectric sensor is installed to be able to align with the electrical connection area of ​​the emergency power supply to monitor abnormal discharge spark signals. Once a discharge phenomenon caused by poor contact or short circuit occurs, the system can issue an early warning for the monitored abnormal spark signal to avoid the influence of light signals in other areas (such as changes in ambient light), thereby improving the accuracy of abnormal discharge monitoring. The current sensor is installed on the output circuit of the emergency power supply and can monitor the output current of the power supply in real time. In particular, when the load changes suddenly, short-circuits or overloads, the sensor can immediately capture the current fluctuations, and the system responds quickly. The voltage sensor is installed on the battery pack terminal of the emergency power supply and can obtain the output voltage of the battery pack in real time. In a battery pack composed of multiple cells, the performance of different cells may vary, resulting in inconsistent voltage. The abnormal total voltage at the monitored terminals helps indicate whether the battery pack needs to be balanced.

[0028] In step S130, data preprocessing is performed on the multi-channel time series sampling data to obtain corresponding online monitoring time series data.

[0029] Here, environmental noise and the interference of the sensor itself can be effectively eliminated through data preprocessing. More specifically, the parameters of different channels are synchronously calibrated to ensure the consistency of the time series. In addition, the time series data is normalized to eliminate the scale differences between different data sources, which is convenient for subsequent feature extraction and model training. The processed data is stored in a preset format to form online monitoring time series data.

[0030] In this embodiment, data preprocessing effectively improves the quality of data and reduces the influence of noise and error. In addition, through synchronous calibration and normalization processing, multi-channel data can more consistently reflect the actual working conditions of the actual battery, providing a reliable basis for subsequent feature extraction.

[0031] In some embodiments, various filtering modules may also be used to perform noise filtering on the time series sampling data, such as adaptive Kalman filtering or multi-stage filtering. In the process of practicing this application, adaptive Kalman filtering is compared with multi-stage filtering. Although the design and implementation of multi-stage filtering (low-pass-high-pass-bandpass filter) are relatively simple, it can effectively filter out the noise of a specific frequency band and retain the effective information of the target frequency band in the signal, but it is only suitable for processing signals with clear spectral characteristics. In particular, for signals whose spectral characteristics change rapidly or whose spectral characteristics are unclear, the effect of multi-stage filtering is not ideal.

[0032] In combination with the fusion scenario of multiple sensing parameters in this embodiment, it is necessary to collect and process sensing time series data in real time, and the signal will show dynamic change characteristics. In addition, the spectrum characteristics of the data collected by various sensors (current, voltage, sound wave, photoelectric) are different, and it is necessary to efficiently filter out noise, enhance signal quality, and improve the accuracy of state detection.

[0033] In some examples of this embodiment, the adaptive Kalman filter can adjust the filtering parameters according to the real-time noise characteristics, adapt to the dynamic changes of the signal, is suitable for processing non-stationary signals, and can provide the best estimate of the signal state. It is especially suitable for processing complex signals with dynamic characteristics, and can effectively filter out noise in the case of strong noise and retain the dynamic characteristics of the signal.

[0034] More specifically, based on the adaptive Kalman filter module, the multi-channel time series sampling data is preprocessed to obtain the corresponding online monitoring time series data: , Formula (1) , Formula (2) , Formula (3) , Formula (4) In the formula, , , and Respectively indicate time The current state estimation value, voltage state estimation value, acoustic state estimation value and photoelectric state estimation value; , , and Respectively indicate time Current observation values, voltage observation values, acoustic wave observation values ​​and photoelectric observation values; , , and They represent the current Kalman gain matrix, the voltage Kalman gain matrix, the acoustic Kalman gain matrix and the photoelectric Kalman gain matrix respectively; , , and They represent the current observation matrix, voltage observation matrix, acoustic wave observation matrix and photoelectric observation matrix respectively; , , and Respectively based on time t -1 information predicts at time t The photoelectric signal state value, voltage signal state value, sound wave signal state value and photoelectric state value.

[0035] Through this embodiment, the adaptive Kalman filter is applied, so that the emergency power supply state monitoring platform can update the state estimation and error covariance matrix in real time, adapt to the dynamic changes of the signal, dynamically adjust the filter parameters according to the real-time noise characteristics, effectively filter out the noise, and improve the signal-to-noise ratio of the signal. As a result, the noise interference in the sampled signal is significantly reduced, making the signal clearer, thereby improving the accuracy of state detection. In addition, through the dynamic adjustment of the Kalman gain, the best estimation of the signal state is achieved, the key features in the sampled signal can be more accurately identified and extracted, and the stability and reliability of the signal are enhanced. In addition, the adaptive Kalman filter can simultaneously process multiple sensor data such as current, voltage, sound waves and photoelectricity, provide the best state estimation for each data source, and through the effective fusion of multimodal data, it can integrate the information of different data sources to improve the comprehensiveness and accuracy of state detection.

[0036] In step S140, a multimodal fusion feature group corresponding to the online monitoring time series data is extracted, and the multimodal fusion feature group includes time domain features, frequency domain features, time-frequency domain features and space domain features.

[0037] In some embodiments, time domain features (such as peak value, mean value, standard deviation, etc.), frequency domain features (such as spectrum distribution, main frequency components, etc.), time-frequency domain features (such as wavelet transform results, time-frequency graphs, etc.) and space domain features (features used for zero-sum evaluation of the correlation between signals from different channels) are extracted from the pre-processed time series data. Thus, the multimodal fusion feature group integrates various features of the sensor sampling signal, provides rich information expression, and improves the accuracy of state prediction.

[0038] Figure 2FIG. 2 shows an operation flow chart of an example of extracting a multimodal fusion feature group according to an embodiment of the present application. Figure 2 As shown, in step S210, time domain features are extracted from the online monitoring time series data, and the time domain features include signal peak value, signal standard deviation and signal mean value. The maximum amplitude of the signal can be reflected by the signal peak value, the size of the signal fluctuation can be reflected by the signal standard deviation, and the overall level of the signal can be reflected by the signal mean value. In step S220, the online monitoring time series data is subjected to Fourier transform to extract the corresponding frequency domain features, and the frequency domain features include the main frequency and spectrum energy. The main frequency can reflect the main frequency component of the signal, and the spectrum energy can reflect the frequency energy distribution of the signal. In step S230, the online monitoring time series data is subjected to wavelet transform to extract the corresponding time-frequency domain features, and the time-frequency domain features include wavelet coefficients and time-frequency energy distribution. The local characteristics of the signal can be reflected by the wavelet coefficients, and the time-frequency characteristics of the signal can be reflected by the time-frequency energy distribution. In step S240, the spatial domain features are calculated according to each channel signal in the online monitoring time series data, and the spatial domain features include channel signal correlation and channel signal covariance matrix. The channel signal correlation can quantitatively reflect the relationship between different channel signals, and the channel signal covariance matrix can reflect the joint changes of the signals.

[0039] More specifically, the wavelet coefficients are calculated as follows: , Formula (5) In the formula, Indicated in scale and displacement The wavelet coefficients under Indicates at time The signal value at Representation scale and displacement The wavelet basis functions under represents the complex conjugate of the wavelet basis function.

[0040] Here, the wavelet transform can analyze the signal at different scales and time displacements, and capture the mutation and transient characteristics of the signal in a short period of time. Some abnormal mutation signals (for example, electric sparks) usually appear as sharp pulses in a short period of time. These transient events can be effectively identified and located through wavelet coefficients. Therefore, the detection sensitivity of the mutation sampling signal is improved, and abnormal events in a short period of time can be accurately identified, which helps to detect potential power supply anomalies at an early stage. In addition, the wavelet transform has multi-scale analysis capabilities, which can simultaneously observe the low-frequency and high-frequency components of the signal and provide full-spectrum signal characteristics. Different types of anomalies may show different spectral characteristics, and wavelet coefficients can help distinguish different types of abnormal signal events. Therefore, the recognition ability of the power supply state recognition model for various abnormal events is enhanced, providing more accurate state recognition.

[0041] The time-frequency energy distribution is calculated by the following formula: , Formula (6) In the formula, represents the time-frequency energy distribution, Indicates at time and frequency The amplitude of the signal time spectrum at .

[0042] Here, the time-frequency energy distribution can simultaneously display the changes in the signal in time and frequency, revealing the time-frequency characteristics of the signal. The sensor sampling signal will have obvious time-frequency characteristics when the UPS emergency power supply operates abnormally, and these characteristics can be intuitively seen through the time-frequency energy distribution. Therefore, the ability to analyze dynamic signals is improved, and the time-frequency distribution of the sensor sampling signal is obtained to help identify complex discharge patterns. In addition, the time-frequency energy distribution can decompose and reconstruct complex signals and separate the energy distribution of different frequency components. For sensor sampling signals containing multiple frequency components, the time-frequency energy distribution can effectively distinguish the energy contribution of each component and support the decomposition and reconstruction of complex signals. Therefore, the processing ability of the power state recognition model for complex signals is enhanced, and discharge signals of multiple frequency components can be separated and identified, thereby improving the recognition accuracy of the power state type.

[0043] The channel signal correlation is calculated by the following formula: , Formula (7) In the formula, Indicates Channels at time The signal value at Indicates Channels at time The signal value at Indicates The mean value of the channel signal, Indicates The mean value of the channel signal, Represents the correlation of channel signals.

[0044] Here, the channel signal correlation reflects the linear relationship between the signals collected by different sensors. By analyzing the correlation of different sensor data, the relationship between different channel signals is revealed. As a result, the comprehensive analysis capability of multi-source signals is improved, the correlation characteristics between different sensor signals can be discovered, and the overall effect of abnormal power state detection can be enhanced. In addition, channel signal correlation can serve as the basis for multi-sensor data fusion, helping the model to integrate the data of each sensor. Different sensors may show different signal characteristics in the same event. Through correlation analysis, this information can be integrated to provide more comprehensive abnormal characteristics and improve the accuracy and reliability of state monitoring.

[0045] The channel signal covariance matrix is ​​calculated by the following formula: , Formula (8) In the formula, Represents the covariance matrix of the channel signal; Indicates the total number of sampling moments; Indicates at time The signal vector at contains the signal values ​​of all channels; represents the mean of the signal vector.

[0046] Here, the channel signal covariance matrix reflects the joint changes of multi-channel signals and provides the synergistic characteristics of multi-sensor signals. Through the covariance matrix, the joint fluctuations of each channel signal can be quantified to reveal the complex relationship between multi-sensor data. As a result, the overall understanding of multi-channel signals is improved, the joint features between signals can be captured, and the comprehensiveness of abnormal state detection can be enhanced. In addition, the covariance matrix provides the statistical characteristics of multi-dimensional features, which helps to establish a statistical model for multi-dimensional signals. By analyzing the covariance of the signal, the main change patterns and abnormal patterns in the multi-sensor data can be identified, which helps to enhance the modeling ability of the power state recognition model.

[0047] In step S150, the power attribute data and the multimodal fusion feature group are input into the power status recognition model to determine the operating health status of the emergency power supply, which includes any one of the following status types: normal state, warning state, restricted operating state and fault state.

[0048] In some embodiments, the power state recognition model adopts a time series classification model based on an attention mechanism. The extracted multimodal fusion feature group and power attribute data are used as inputs and input into the power state recognition model. After training and optimization, the time series classification model based on the attention mechanism can output the state type and confidence of the emergency power supply according to the input data, can fully mine the implicit patterns and associations in the input data, and improve the accuracy of the prediction results of the power supply operation status.

[0049] In this embodiment, the power attribute data is included in the input feature dimension of the power state identification model, which includes information such as power capacity, power rated power, battery internal resistance, historical charge and discharge cycle data, and historical abnormal records. These data can provide comprehensive background information of the emergency power supply, which helps the model to better understand the context of the current data. According to the different attributes of the emergency power supply, the model can perform more personalized operation status analysis and improve the pertinence and accuracy of the prediction. In addition, the rich feature diversity helps to improve the generalization ability of the model under different emergency power supplies and different working conditions.

[0050] It should be noted that the warning state, restricted operation state and fault state can be used to indicate different degrees of abnormality of the emergency power supply, which can be used to express corresponding response strategies.

[0051] Table 1 describes various types of power supply abnormal status.

[0052]

[0053] In the embodiment of the present application, a multi-level classification is performed for abnormal power supply status, so that the system can gradually upgrade the response strategy when facing different degrees of abnormality: from monitoring and early warning to partial restricted operation, and then to complete shutdown and fault handling. In this way, not only the safety and stability of the emergency power supply are improved, but also clear maintenance guidance and response strategies are provided for operation managers, effectively reducing the risk of unexpected shutdown.

[0054] Figure 3 A schematic diagram of the structural connection of an example of a power state identification model according to an embodiment of the present application is shown.

[0055] More specifically, the power state recognition model 300 includes a cascaded input layer 310 , a feature extraction layer 320 , a self-attention mechanism layer 330 , a hybrid network layer 340 , and an output layer 350 .

[0056] The input layer 310 is used to receive the multimodal fusion feature group and power source attribute data.

[0057] Specifically, the input layer is used to receive the multimodal fusion feature group and power attribute data, and perform standardization processing to make the mean and variance of the features consistent to ensure the stability of the model.

[0058] In this embodiment, the multimodal feature fusion of time domain, frequency domain, time-frequency domain and space domain fully captures the different characteristics of the sensor sampling signal, and integrates the power attribute data of the emergency power supply, effectively enhancing the ability to identify the abnormal state of the power supply. As a result, it is possible to more accurately identify various types of emergency power supply states, especially improve the detection accuracy of abnormal states, so that in a low signal-to-noise ratio environment, it is still possible to effectively distinguish abnormal mode feature signals and noise, and reduce the false alarm rate and missed alarm rate.

[0059] The feature extraction layer 320 is used to extract deep features through shared convolution kernels to capture the complex patterns and potential relationships of the signal.

[0060] In some embodiments, the feature extraction layer uses a multimodal convolutional neural network (MCNN). MCNN combines data inputs of different modalities and performs feature extraction by sharing convolution kernels to improve feature fusion effects.

[0061] More specifically, the convolution operation is performed in the following way: , Formula (9) In the formula, For the convolution kernels, is the bias term, is the activation function (such as ReLU), represents the convolution operation, Represents the input features of the feature extraction layer.

[0062] Here, the convolution kernel Pooling functions (such as max pooling or average pooling) are used to downsample feature maps. . Then, through multi-layer convolution and pooling operations, deep features are extracted It should be understood that the model layer structure described in the above formula is only for example, and other types of structures may also be used, such as a more novel improved structure. More details will be described below in conjunction with other examples.

[0063] The self-attention mechanism layer 330 is used to weight features through the self-attention mechanism, dynamically assign importance weights to features of each modality, and enhance the influence of key features.

[0064] Here, the attention mechanism can be used to strengthen the recognition of specific patterns in sensor sampling signals and improve the accuracy of the model. Through the self-attention mechanism layer, the model's ability to capture key features is improved, the richness and accuracy of feature representation are enhanced, and it can identify important patterns in signals and improve the sensitivity of anomaly detection.

[0065] The hybrid network layer 340 is used to adopt a hybrid structure of Transformer and Long Short-Term Memory (LSTM) network to further extract temporal features and global features.

[0066] Here, Transformer is used to capture long-distance dependencies and global features to improve the representation ability of the model. In addition, LSTM is used to capture time series features and dynamic changes to enhance the model's ability to process time series data. As a result, the hybrid structure can process global and local time series features at the same time, improving the accuracy and timeliness of anomaly detection.

[0067] The output layer 350 maps the deep features to the state category space through a fully connected layer, and outputs the state type and confidence through a softmax classifier to determine the operating health status of the emergency power supply.

[0068] Specifically, the fully connected layer uses Dropout and regularization techniques to prevent overfitting of the model and improve the generalization ability of the model. In addition, the Softmax classifier can output the probability distribution of power state types and provide confidence information to facilitate further decision-making and analysis, thereby improving the response speed to real-time online monitoring results.

[0069] In some examples of the embodiments of the present application, the feature extraction layer includes a residual connection module and a multi-scale convolution module.

[0070] The residual connection module can introduce shortcuts between convolutional layers so that the input can be directly added to the output, effectively alleviating the gradient vanishing problem.

[0071] , Formula (10) In the formula, For the convolution kernels, is the bias term, Indicates The output of the convolution layer, Represents the batch normalization function.

[0072] The multi-scale convolution module can capture feature information of different scales through convolution kernels of different sizes, thereby enhancing the ability to recognize various feature patterns.

[0073] , Formula (11) In the formula, represents multi-scale convolution features, represents the feature vector generated by the 3x3 convolution kernel, represents the feature vector generated by the 5x5 convolution kernel, Represents the feature vector generated by the 7x7 convolution kernel.

[0074] In this embodiment, by using convolution kernels of different sizes (such as , , ), the feature extraction layer can capture the characteristic information of the signal at different scales. As a result, the details and global features of the power parameter characteristics can be fully covered, richer feature information can be extracted, and the effectiveness and accuracy of anomaly detection can be improved. In addition, the residual connection helps solve the gradient vanishing problem in deep networks, allowing information to be more effectively transmitted between network layers, maintaining the integrity of information during feature extraction, avoiding information loss, and enhancing the accuracy and stability of feature extraction.

[0075] In some examples of the embodiments of the present application, the self-attention mechanism layer adopts a multi-head self-attention layer based on layer normalization, and its structure is: , Formula (12) , Formula (13) , Formula (14) , Formula (15) , Formula (16) In the formula, represents the input feature matrix, Indicates The query matrix, key matrix and value matrix of each head, Indicates The weight matrix corresponding to each head; represents the scaling factor, Indicates The attention score matrix of each head, Indicates The weighted sum matrix of the heads; represents the feature concatenation function; Represents the output weight matrix of multi-head attention; Representation layer normalization function, represents the fusion output features of all heads, Represents the output feature matrix of the self-attention mechanism layer.

[0076] In this embodiment, the multi-head self-attention mechanism is used to capture different aspects of the input features in parallel, and multiple attention heads are used to simultaneously focus on different feature dimensions and patterns. Thus, the diversity and richness of the features are improved, and the model's ability to recognize complex signals is enhanced. In addition, through the joint action of multiple attention heads, the model can more accurately extract the key features in the power parameter features and improve the accuracy of state detection. In addition, the multi-head self-attention mechanism captures different aspects of the input features by calculating multiple attention heads in parallel, thereby improving the model's adaptability to different signal patterns. Thus, the adaptability of the model in different operating environments and working conditions is improved, the ability to process unknown signals and abnormal signals is enhanced, and the robustness and generalization ability of the power state recognition model can be improved, so that it can maintain good performance in a variety of scenarios.

[0077] Furthermore, layer normalization performs standardization processing on the input of each layer, so that the mean of each layer input is 0 and the variance is 1, reducing the change of internal covariates. In this way, the model training is accelerated, the instability caused by the change of internal covariates during the training process is reduced, the convergence speed of the model can be improved, and the power state recognition model can respond quickly in practical applications.

[0078] Figure 4 A schematic diagram of the structural connection of an example of a hybrid network layer according to an embodiment of the present application is shown.

[0079] like Figure 4 As shown, the hybrid network layer 400 includes a convolutional layer 410, a bidirectional LSTM layer 420 and an attention layer 430.

[0080] Here, the convolutional layer is used to extract local features and reduce feature dimensions, the bidirectional LSTM layer (Bi-LSTM) is used to capture bidirectional temporal features, and the attention mechanism is used to weight the LSTM output to increase the influence of key features.

[0081] More specifically, the convolutional layer 410 is used to extract local features and reduce feature dimensions: , Formula (17) In the formula, represents the output feature map of the convolutional layer, represents the convolution function, represents the activation function, Represents the bias term.

[0082] By sharing convolution kernels to extract local features, reduce feature dimensions, and enhance feature representation capabilities, the important features in the multi-dimensional feature signals of the power supply can be effectively captured, the accuracy of feature representation can be improved, and the accuracy of the power supply status recognition model can be enhanced.

[0083] The bidirectional LSTM layer 420 is used to capture bidirectional time series features: , Formula (18) In the formula, Indicates at time The output hidden state of Indicates at time The hidden state of the forward LSTM is Indicates at time The hidden state of the reverse LSTM, Represents a bidirectional LSTM function.

[0084] Bidirectional LSTM captures bidirectional time series features through memory and forgetting mechanisms, and is suitable for processing time series data. As a result, it can capture the bidirectional time series information of the multi-dimensional characteristic signal of the power supply, comprehensively extract the time series features, and enhance the model's ability to understand time dependencies.

[0085] The attention layer 430 is used to perform attention weighted calculation on the output features of the bidirectional LSTM layer: , Formula (19) , formula (20) , Formula (21) In the formula, represents the calculated intermediate value of the attention weight, represents the attention weight vector, represents the attention weight matrix, and Indicates attention bias; represents the context vector, Indicates at time The attention weight.

[0086] Thus, the output of LSTM is weighted by the attention layer 430 to increase the influence of key features and enhance the model's focus on important features. Thus, key time series features are identified and enhanced, and the model's sensitivity to key abnormal features is increased, which can improve the model's adaptability in different operating environments and working conditions and enhance the robustness and generalization performance of abnormal state detection.

[0087] In the embodiment of the present application, by using the enhanced hybrid network layer of convolutional layer, bidirectional LSTM layer and attention mechanism, the performance of the power state recognition model can be significantly improved, the model feature extraction and representation capabilities can be enhanced, the accuracy of state detection can be improved, and key features can be identified and strengthened through the attention mechanism. Thus, by comprehensively capturing various features of the multi-dimensional parameters of the emergency power supply, the depth and breadth of anomaly detection are enhanced.

[0088] In some examples of the embodiments of the present application, in order to ensure the prediction performance of the power state recognition model, it is particularly necessary to pay attention to the design details of the loss function of the power state recognition model. Here, by adopting a comprehensive loss function, the cross entropy loss and the weighted cross entropy loss are integrated to better meet the analysis requirements of the abnormal state of emergency equipment.

[0089] More specifically, the loss function of the power state recognition model is for: , Formula (22) , Formula (23) , Formula (24) , Formula (25) , Formula (26) In the formula, and Represent the cross entropy loss term and the weighted cross entropy loss term, respectively. and Respectively represent the adaptive weight coefficients of the corresponding loss terms; Indicates a preset constant, such as a small constant to prevent division by zero; represents the total number of samples in the data sample set, Indicates The actual labels of samples, Indicates The label prediction probability of each sample; Separate The weight of the samples.

[0090] It should be noted that the difference between the predicted probability distribution and the actual label distribution is calculated based on the cross entropy loss. By maximizing the probability difference between faulty and non-faulty states, the detection accuracy of faulty states is improved and the possibility of false positives and negatives is reduced. Based on the weighted loss, different weights are assigned to different samples. By assigning higher weights to fault types, the model can detect the characteristic patterns of faults more keenly, making the model more effective in handling high-priority fault states in practical applications.

[0091] In this embodiment, an adaptive weight adjustment mechanism is selected in the loss function. It should be noted that the adaptive weight adjustment mechanism is a method of dynamically adjusting the weights of each loss function. The purpose is to automatically adjust the weights of each loss function according to the performance of the model on different tasks during the training process to optimize the overall performance of the model. Therefore, during the training process, the weights are automatically adjusted according to the current loss to ensure that each loss term is properly optimized, so that the model can show strong robustness and adaptability in a variety of different environments and equipment conditions.

[0092] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of actions combined, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0093] Figure 5 A structural block diagram of an example of an emergency power supply status monitoring system based on multimodal data according to an embodiment of the present application is shown.

[0094] like Figure 5 As shown, the emergency power supply status monitoring system 500 based on multimodal data includes an attribute data acquisition unit 510, a sensor data acquisition unit 520, a data preprocessing unit 530, a multimodal feature extraction unit 540 and an operation status identification unit 550.

[0095] The attribute data acquisition unit 510 is used to acquire the power attribute data of the emergency power supply; the power attribute data includes power capacity, power rated power, battery internal resistance, historical charge and discharge cycle data and historical abnormality records.

[0096] The sensor data acquisition unit 520 is used to collect multi-channel timing sampling data of the emergency power supply corresponding to a preset sampling time period in real time based on a sensor group; the sensor group includes a current sensor, a voltage sensor, an acoustic wave sensor and a photoelectric sensor; the multi-channel timing sampling data includes current timing parameters, voltage timing parameters, acoustic wave timing parameters and light sensing timing parameters.

[0097] The data preprocessing unit 530 is used to perform data preprocessing on the multi-channel time series sampling data to obtain corresponding online monitoring time series data.

[0098] The multimodal feature extraction unit 540 is used to extract a multimodal fusion feature group corresponding to the online monitoring time series data; the multimodal fusion feature group includes time domain features, frequency domain features, time-frequency domain features and space domain features.

[0099] The operating status identification unit 550 is used to input the power supply attribute data and the multimodal fusion feature group into the power supply status identification model to determine the operating health status of the emergency power supply; the operating health status includes any one of the following status types: normal state, warning state, restricted operating state and fault state.

[0100] In some embodiments, an embodiment of the present application provides a non-volatile computer-readable storage medium, in which one or more programs including execution instructions are stored, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute the steps of any of the above-mentioned emergency power supply status monitoring methods based on multimodal data in the present application.

[0101] In some embodiments, the embodiments of the present application also provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer performs any step of the above-mentioned emergency power supply status monitoring method based on multimodal data.

[0102] In some embodiments, the embodiments of the present application also provide an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of an emergency power supply status monitoring method based on multimodal data.

[0103] Figure 6 is a schematic diagram of the hardware structure of an electronic device for executing an emergency power supply status monitoring method based on multimodal data provided by another embodiment of the present application, such as Figure 6 As shown, the device includes: One or more processors 610 and memory 620, Figure 6 A processor 610 is taken as an example.

[0104] The device for executing the emergency power supply status monitoring method based on multimodal data may further include: an input device 630 and an output device 640 .

[0105] The processor 610, the memory 620, the input device 630 and the output device 640 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0106] The memory 620, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the emergency power supply status monitoring method based on multimodal data in the embodiment of the present application. The processor 610 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 620, that is, the emergency power supply status monitoring method based on multimodal data in the above method embodiment is implemented.

[0107] The memory 620 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 620 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 620 may optionally include a memory remotely arranged relative to the processor 610, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0108] The input device 630 can receive input digital or character information and generate signals related to user settings and function control of the electronic device. The output device 640 can include a display device such as a display screen.

[0109] The one or more modules are stored in the memory 620, and when executed by the one or more processors 610, the emergency power supply status monitoring method based on multimodal data in any of the above method embodiments is executed.

[0110] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0111] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: (1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions and its main purpose is to provide voice and data communications. This type of terminal includes: smart phones, multimedia phones, functional phones, and low-end phones.

[0112] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, etc.

[0113] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0114] (4) Other onboard electronic devices with data interaction functions, such as on-board devices installed in vehicles.

[0115] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for monitoring emergency power supply status based on multimodal data, comprising: Obtain power source attribute data of the emergency power source; The power source attribute data includes power source capacity, power source rated power, battery internal resistance, historical charge and discharge cycle data and historical abnormal records; Based on the sensor group, multi-channel time series sampling data corresponding to the preset sampling time period of the emergency power supply is collected in real time; the sensor group includes a current sensor, a voltage sensor, an acoustic wave sensor and a photoelectric sensor; the multi-channel time series sampling data includes current timing parameters, voltage timing parameters, acoustic wave timing parameters and light sensing timing parameters; Performing data preprocessing on the multi-channel time series sampling data to obtain corresponding online monitoring time series data; Extracting a multimodal fusion feature group corresponding to the online monitoring time series data; the multimodal fusion feature group includes time domain features, frequency domain features, time-frequency domain features and space domain features; The power source attribute data and the multimodal fusion feature group are input into a power source status recognition model to determine the operating health status of the emergency power source; the operating health status includes any one of the following status types: normal state, warning state, restricted operating state and fault state; the power source status recognition model adopts a time series classification model based on an attention mechanism.

2. The method according to claim 1, wherein: The current sensor is installed on the output circuit of the emergency power supply; the voltage sensor is installed on the battery pack terminal of the emergency power supply; the acoustic wave sensor is installed to the inner surface of the shell of the emergency power supply; the photoelectric sensor is installed to be able to align with the electrical connection area of ​​the emergency power supply to monitor abnormal discharge spark signals.

3. The method according to claim 1, wherein: The data preprocessing of the multi-channel time series sampling data to obtain corresponding online monitoring time series data includes: Based on the adaptive Kalman filter module, the multi-channel time series sampling data is preprocessed to obtain corresponding online monitoring time series data: ， ， ， ， In the formula, , , and Respectively indicate time The current state estimation value, voltage state estimation value, acoustic state estimation value and photoelectric state estimation value; , , and Respectively indicate time Current observation values, voltage observation values, acoustic wave observation values ​​and photoelectric observation values; , , and They represent the current Kalman gain matrix, the voltage Kalman gain matrix, the acoustic Kalman gain matrix and the photoelectric Kalman gain matrix respectively; , , and They represent the current observation matrix, voltage observation matrix, acoustic wave observation matrix and photoelectric observation matrix respectively; , , and Respectively based on time t -1 information predicts at time t The photoelectric signal state value, voltage signal state value, sound wave signal state value and photoelectric state value.

4. The method according to claim 1, wherein: The extracting of a multimodal fusion feature group corresponding to the online monitoring time series data includes: Extracting time domain features from the online monitoring time series data, wherein the time domain features include a signal peak value, a signal standard deviation, and a signal mean value; Performing Fourier transform on the online monitoring time series data to extract corresponding frequency domain features, wherein the frequency domain features include main frequency and spectrum energy; Performing wavelet transform on the online monitoring time series data to extract corresponding time-frequency domain features, wherein the time-frequency domain features include wavelet coefficients and time-frequency energy distribution; The wavelet coefficients are calculated by the following formula: ， In the formula, Indicated in scale and displacement The wavelet coefficients under Indicates at time The signal value at Representation scale and displacement The wavelet basis functions under represents the complex conjugate of the wavelet basis function; The time-frequency energy distribution is calculated by the following formula: ， In the formula, represents the time-frequency energy distribution, Indicates at time and frequency The amplitude of the signal spectrum at ; Calculate spatial domain features according to each channel signal in the online monitoring time series data, wherein the spatial domain features include channel signal correlation and channel signal covariance matrix; The channel signal correlation is calculated by the following formula: ， In the formula, Indicates Channels at time The signal value at Indicates Channels at time The signal value at Indicates The mean value of the channel signal, Indicates The mean value of the channel signal, Represents the correlation of channel signals; The channel signal covariance matrix is ​​calculated by the following formula: ， In the formula, Represents the covariance matrix of the channel signal; Indicates the total number of sampling moments; Indicates at time The signal vector at contains the signal values ​​of all channels; represents the mean of the signal vector.

5. The method according to any one of claims 1 to 4, wherein: The power state recognition model comprises a cascaded input layer, a feature extraction layer, a self-attention mechanism layer, a hybrid network layer and an output layer; The input layer is used to receive the multimodal fusion feature group and the power source attribute data; The feature extraction layer is used to extract deep features through shared convolution kernels to capture complex patterns and potential relationships of features; The self-attention mechanism layer is used to weight features through the self-attention mechanism, dynamically assign importance weights to features of each modality, and enhance the influence of key features; The hybrid network layer is used to adopt a hybrid structure of Transformer and long short-term memory network LSTM to further extract temporal features and global features; The output layer maps the deep features to the state category space through the fully connected layer, and outputs the state type and confidence through the softmax classifier to determine the corresponding running health state.

6. The method according to claim 5, wherein the feature extraction layer comprises a residual connection module and a multi-scale convolution module.

7. The method according to claim 6, wherein: The self-attention mechanism layer adopts a multi-head self-attention layer based on layer normalization, and its structure is: ， ， ， ， ， In the formula, represents the input feature matrix, Indicates The query matrix, key matrix and value matrix of each head, Indicates The weight matrix corresponding to each head; represents the scaling factor, Indicates The attention score matrix of each head, Indicates The weighted sum matrix of the heads; represents the feature concatenation function; Represents the output weight matrix of multi-head attention; Representation layer normalization function, represents the fusion output features of all heads, Represents the output feature matrix of the self-attention mechanism layer.

8. The method according to claim 7, wherein: The hybrid network layer includes a convolutional layer, a bidirectional LSTM layer and an attention layer; The convolutional layer is used to extract local features and reduce feature dimensions: ， In the formula, represents the output feature map of the convolutional layer, represents the convolution function, represents the activation function, represents the bias term; The bidirectional LSTM layer is used to capture bidirectional time series features: ， In the formula, Indicates at time The output hidden state of Indicates at time The hidden state of the forward LSTM is Indicates at time The hidden state of the reverse LSTM, Represents a bidirectional LSTM function; The attention layer is used to perform attention weighted calculation on the output features of the bidirectional LSTM layer: ， ， ， In the formula, Indicates for The calculated intermediate value of the attention weight, represents the attention weight vector, represents the attention weight matrix, and Indicates attention bias; represents the context vector, Indicates at time The attention weight.

9. The method according to claim 5, wherein: The loss function of the power state recognition model is for: ， ， ， ， ， In the formula, and Represent the cross entropy loss term and the weighted cross entropy loss term, respectively. and Respectively represent the adaptive weight coefficients of the corresponding loss terms; Indicates a preset constant; represents the total number of samples in the data sample set, Indicates The actual labels of samples, Indicates The label prediction probability of each sample; Separate The weight of the samples.

10. An emergency power supply status monitoring system based on multimodal data, comprising: An attribute data acquisition unit, used to acquire power attribute data of the emergency power supply; The power source attribute data includes power source capacity, power source rated power, battery internal resistance, historical charge and discharge cycle data and historical abnormal records; A sensor data acquisition unit is used to collect multi-channel time series sampling data of the emergency power supply corresponding to a preset sampling time period in real time based on a sensor group; the sensor group includes a current sensor, a voltage sensor, an acoustic wave sensor and a photoelectric sensor; the multi-channel time series sampling data includes a current time series parameter, a voltage time series parameter, an acoustic wave time series parameter and a photosensitive time series parameter; A data preprocessing unit, used for performing data preprocessing on the multi-channel time series sampling data to obtain corresponding online monitoring time series data; A multimodal feature extraction unit, used to extract a multimodal fusion feature group corresponding to the online monitoring time series data; the multimodal fusion feature group includes time domain features, frequency domain features, time-frequency domain features and space domain features; An operating status identification unit is used to input the power supply attribute data and the multimodal fusion feature group into a power supply status identification model to determine the operating health status of the emergency power supply; the operating health status includes any one of the following status types: normal state, warning state, restricted operating state and fault state; the power supply status identification model adopts a time series classification model based on an attention mechanism.

Images