Electric automation system fault detection method
By combining multimodal fusion methods of sensor signals, video and audio data, the problems of sensor location selection and detection cycle influence are solved, and high accuracy of fault detection in electrical automation systems is achieved, with an average accuracy rate of 95%.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINESE PEOPLES LIBERATION ARMY ARMY SERVICES UNIVERSITY
- Filing Date
- 2025-12-25
- Publication Date
- 2026-04-10
AI Technical Summary
Existing fault detection strategies in electrical automation systems rely on empirical selection of sensor locations, leading to uncertainty in detection results and impacting detection accuracy due to the long detection cycle. Intermittent sensor detection also affects timeliness.
Fault detection is performed by combining sensor signals with video and audio data. Fault identification is performed by using RNN, CNN and ResNet18 networks through feature extraction and multimodal fusion. An attention-based multimodal fusion method is adopted to improve detection accuracy.
It improves the accuracy of fault detection in electrical automation systems, with an average accuracy rate of over 95%, which is superior to detection methods that rely solely on sensor signals.
Smart Images
Figure CN121832508A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of fault diagnosis of electrical automation systems, and particularly relates to a fault detection method of an electrical automation system. BACKGROUND
[0002] In the field of industrial production, the reliability and safety of electrical automation systems are the key to ensuring production continuity and reducing risks. System failures can cause serious production interruptions, economic losses, and even casualties. Therefore, effective fault detection technology is of great significance to improve the stability and reliability of the system.
[0003] However, some existing fault detection strategies for electrical automation systems are to arrange sensors at different positions of the electrical automation system, and to monitor faults through data collected by the sensors. Since the sensors are arranged in a scattered manner, the selection of sensor positions can affect the accuracy and timeliness of fault detection. Moreover, the selection of sensor positions often depends on the experience of designers, which may introduce uncertainty to the results of fault detection. In addition, the detection of sensors is intermittent, and the length of the detection cycle can also affect the detection results. SUMMARY
[0004] The purpose of the present application is to provide a fault detection method for an electrical automation system, which can combine sensor detection signals with video and audio to judge the faults of the electrical automation system, and improve the accuracy of fault detection of the electrical automation system.
[0005] The technical solution provided by the present application is as follows:
[0006] A fault detection method for an electrical automation system, comprising the following steps:
[0007] Step 1: Obtain temperature and vibration signals at multiple positions in the electrical automation system according to a sampling period, and obtain continuous video and continuous audio during the operation of the electrical automation system;
[0008] Step 2: Divide the continuous video and the continuous audio into multiple video segments and audio segments according to the sampling period; and group the temperature, vibration signals, video segments and audio segments of the same sampling period into a sample to obtain samples of multiple sampling periods;
[0009] Step 3: Extract features from the temperature, vibration signals, video segments and audio segments in each sample to obtain temperature feature vectors, vibration feature vectors, video feature vectors and audio feature vectors of the samples;
[0010] Step four, multi-modal fusion is performed on the video feature vector and the audio feature vector of the same sample to obtain a fusion feature vector, and the fusion feature vector is combined with the temperature feature vector and the vibration feature vector into a multi-channel tensor;
[0011] Step five, fault recognition is performed on the multi-channel tensor input fault detection network model to obtain an electrical automation system fault type.
[0012] Preferably, the temperature and vibration signals are respectively subjected to feature extraction by an RNN model.
[0013] Preferably, video feature extraction is performed by a CNN model.
[0014] Preferably, audio feature extraction is performed by a ResNet18 network.
[0015] Preferably, a multi-modal fusion method based on an attention mechanism is adopted to perform multi-modal fusion on the video feature vector and the audio feature vector of the same sample to obtain a fusion feature vector of each sample, including the following steps:
[0016] The video feature vector and the audio feature vector are respectively subjected to a layerNorm layer to obtain a video query vector Q v , an audio key vector K A , and an audio value vector V A .
[0017] Q v , K A , and V A are input into a multi-head cross-attention module to obtain an output feature vector A x .
[0018] Q v is added to A x , and the addition result is regularized to obtain a regularized vector.
[0019] The regularized vector is subjected to a forward propagation layer to obtain a forward propagation vector.
[0020] The regularized vector and the forward propagation vector are added to obtain the fusion feature vector.
[0021] Preferably, the fault detection network model comprises:
[0022] a plurality of convolution layers;
[0023] a plurality of pooling layers, which are one-to-one correspondingly arranged with the convolution layers and connected at output ends of the convolution layers; and
[0024] a fully connected layer connected at an output end of the last pooling layer.
[0025] Preferably, before the step two, it further includes cleaning the temperature and vibration signals, removing errors and outliers.
[0026] Preferably, before the step three, it further includes:
[0027] The noise and redundant information in the audio are reduced by the method of pre-emphasis, framing and windowing.
[0028] The beneficial effects of the present application are:
[0029] The electrical automation system fault detection method provided by the present application can improve the accuracy of electrical automation system fault detection by combining sensor detection signals with video and audio to judge the electrical automation system fault. BRIEF DESCRIPTION OF DRAWINGS
[0030] Figure 1 The flowchart of the electrical automation system fault detection method provided by the present application. DETAILED DESCRIPTION
[0031] The present application will be further described in detail below, so that those skilled in the art can implement it according to the description.
[0032] As shown in the accompanying drawings, the present application provides an electrical automation system fault detection method, and the specific implementation process is as follows. Figure 1
[0033] I. According to the sampling period, the temperature and vibration signals of multiple positions in the electrical automation system are obtained, and the continuous video and continuous audio during the operation of the electrical automation system are obtained.
[0034] Due to the different number and types of devices included in different electrical automation systems, the sensors need to be set according to the characteristics of the electrical automation system itself. Common sensors include temperature sensors, current and voltage sensors, vibration sensors, etc. However, it has been found through research that in electrical automation systems, the changes in current and voltage are generally related to temperature changes. Therefore, only temperature and vibration signals are used as detection parameters in the present application to avoid redundancy and mutual influence caused by too many data types. In the present application, multiple temperature sensors and multiple vibration sensors are set in an electrical automation system. Among them, the temperature sensors are set at devices prone to overheating (such as motors, frequency converters), and the vibration sensors are installed at rotating machinery or bearings and other components.
[0035] As a preferred, more than three sensors are installed at the same position to ensure that the data of the position will not be lost when the sensor fails. If the data collected by the sensors at the same position are all normal, the average value of the values collected by the multiple sensors at the position is taken as the temperature or vibration signal data of the position.
[0036] In addition, due to the limited installation positions of the temperature sensor and the vibration sensor, some positions may be missed, resulting in that the detection system cannot timely detect the failure problem. It is found through research that when the electrical automation system fails, the dynamic (action) change or abnormal sound of the equipment often occurs at the same time. Therefore, the running video and audio of the electrical automation system are combined with the temperature and vibration signals to judge the failure.
[0037] II. The continuous video and the continuous audio are divided into multiple video segments and audio segments according to the sampling period; and the temperature, vibration signal, video segment and audio segment of the same sampling period are combined into a sample, and samples of multiple sampling periods are obtained.
[0038] In an embodiment, the sampling period is set to 3s.
[0039] As a preferred, before the sample is formed, the temperature or vibration signal data is also cleaned, and the cleaning process specifically includes removing errors and abnormal values.
[0040] III. The temperature, vibration signal, video segment and audio segment in each sample are respectively subjected to feature extraction, and the temperature feature vector, vibration feature vector, video feature vector and audio feature vector of each sample are obtained.
[0041] Before the feature extraction is performed, it also includes reducing noise and redundant information in the audio through pre-emphasis, frame division and windowing.
[0042] As a preferred, in the present application, the temperature and vibration signal are respectively subjected to feature extraction through an RNN model; the video feature extraction is performed through a CNN model; and the audio feature extraction is performed through a ResNet18 network.
[0043] The audio feature extraction is performed by taking ResNet18 as a basic feature learning network structure, wherein the audio feature extraction is to convert the audio signal into a mel frequency cepstral coefficient (MFCC) and generate an audio feature vector.
[0044] IV. The video feature vector and the audio feature vector of the same sample are subjected to multi-modal fusion, a fusion feature vector is obtained, and the fusion feature vector is combined with the temperature feature vector and the vibration feature vector into a multi-channel tensor.
[0045] As a preferred, a multi-modal fusion strategy based on attention mechanism is adopted, and the obtained multi-modal fusion features are as follows.
[0046] (1) A multi-modal fusion strategy model based on cross-attention mechanism of transformer is constructed, and the video feature vector X v and the audio feature vector X A are extracted as two inputs of the multi-modal fusion strategy model.
[0047] (2) The video feature vector X v passes through the layerNorm layer to obtain Q v ; the audio feature vector X A passes through the layerNorm layer to obtain K A and V A ; Q v , K A and V A pass through the multi-head cross-attention mechanism to obtain the output feature vector A x .
[0048] Wherein, Q v =X v W Q , K A =V A =X A W K ;
[0049]
[0050] In the formula, X v is the video feature vector, Q v is the video query vector, X A is the audio feature vector, K A is the audio key vector, and V A is the audio value vector. W Q and W K are learnable parameter matrices, is a matrix set, d A is the dimension of the audio feature set, d V is the dimension of the value set, and d k is the dimension of the key set.
[0051] (3) The video query vector Q v and the output feature vector A xThe addition result is regularized to obtain a regularization result; after the regularization result is forward propagated, a forward propagation result is obtained; the regularization result and the forward propagation result are added to obtain a final fusion feature vector A * . Wherein, the video query vector Q v is added to the output feature vector A x . The addition can make the feature vector pass through the deep network, and then be regularized to improve the stability and convergence speed of the model, and then be forward propagated to increase the nonlinearity of the feature and enhance the expression ability thereof.
[0052] The video feature is taken as the Q value (query vector) to generate the feature, the audio feature is taken as the K (key vector) and V (value vector) to generate the feature, and after the multi-head cross attention mechanism, the video feature is cross-modally residual fused with the fusion feature; in this way, the information loss during the fusion of the two modalities can be reduced, and the complementary information of the two can be maximally utilized, and the defect that the video feature as the Q value reduces the transmission of the feature information can be avoided.
[0053] Five, the fault recognition is performed on the multi-channel tensor input fault detection network model to obtain the fault type of the electrical automation system.
[0054] As a preferred, the fault detection network model is composed of a plurality of convolutional layers, a plurality of pooling layers and a fully connected layer. The number of convolutional layers is at least 3, the number of pooling layers is the same as that of the convolutional layers, and the pooling layers are arranged one by one corresponding to the convolutional layers, and the pooling layers are connected to the output ends of the convolutional layers; the fully connected layer is connected to the output end of the last pooling layer.
[0055] The convolutional layers are all 2D convolutional layers with a convolution kernel of 3x3. The fully connected layer includes a ReLU activation function and a softmax function, and the probability of each fault is output by the softmax function. The fault type with the largest probability is taken as the fault type of the electrical automation system.
[0056] Embodiment
[0057] In this embodiment, the fault of an automation production line is detected, and the production line is used for manufacturing electronic products. A video acquisition device can completely capture the running state of the entire production line. In the test, several fault conditions such as device fault, process abnormality and sensor data abnormality are simulated. The collected data is generally taken as a training sample to train the fault detection network model, and the other half is taken as verification data to compare whether the actual fault and the detection result of the trained fault detection network model are consistent.
[0058] In the experiment, data of three consecutive days were collected to simulate the long-time running production line. At the same time, the detection method using only temperature sensors and vibration sensors was used as a control group: the detection method was to splice the temperature and vibration signals after feature extraction to obtain a splicing vector, which was input into the detection network with the same structure as the application for identification. Due to the difference in the data used for training, although the two models have the same framework, the network parameters after training are different.
[0059] By comparing the actual occurring faults with the automatic diagnosis results, the accuracy, recall rate and F1 score were calculated. The results show that the average accuracy of various fault types based on the fault detection network model provided by the application through the combination of sensor detection signals, video and audio is more than 95%. When only sensor signals are used for fault judgment, the average accuracy of various fault types can only reach about 90%.
[0060] This fully shows that the electrical automation system fault detection method provided by the application can improve the accuracy of electrical automation system fault detection by combining sensor detection signals, video and audio to judge the electrical automation system fault.
[0061] Although the embodiments of the application have been disclosed as above, it is not limited to the application listed in the specification and embodiments, and can be fully applied to various fields suitable for the application. For those skilled in the art, other modifications can be easily realized, and therefore the application is not limited to specific details and the figures shown and described herein.
Claims
1. An electrical automation system fault detection method, characterized by, The method comprises the following steps: Step one, obtaining temperature and vibration signals of multiple positions in the electrical automation system according to a sampling period, and obtaining continuous video and continuous audio in the working process of the electrical automation system; Step two, dividing the continuous video and the continuous audio into multiple video segments and audio segments according to the sampling period, and grouping temperature, vibration signals, video segments and audio segments of the same sampling period into a sample to obtain samples of multiple sampling periods; Step three, extracting features of temperature, vibration signals, video segments and audio segments in each sample respectively to obtain temperature feature vectors, vibration feature vectors, video feature vectors and audio feature vectors of each sample; Step four, performing multi-modal fusion on the video feature vectors and the audio feature vectors of the same sample to obtain a fusion feature vector, and combining the fusion feature vector with the temperature feature vector and the vibration feature vector into a multi-channel tensor; Step five, inputting the multi-channel tensor into a fault detection network model to perform fault recognition to obtain a fault type of the electrical automation system.
2. The method of claim 1, wherein, The temperature and vibration signals are respectively extracted by an RNN model.
3. The method of claim 2, wherein, The video features are extracted by a CNN model.
4. The method of claim 2, wherein, The audio features are extracted by a ResNet18 network.
5. The method of claim 4, wherein, A multi-modal fusion method based on an attention mechanism is used to perform multi-modal fusion on the video feature vectors and the audio feature vectors of the same sample to obtain a fusion feature vector of each sample, comprising the following steps: The video feature vector and the audio feature vector are respectively subjected to a layerNorm layer to obtain a video query vector Q v , an audio key vector K A , and an audio value vector V A ; Q v , K A , and V A are input into the multi-head cross attention module to obtain an output feature vector A x ; Q v is added to A x and the result is regularized to obtain a regularized vector; The regularization vector passes through a forward propagation layer to obtain a forward propagation vector; The regularization vector and the forward propagation vector are added to obtain the fusion feature vector.
6. The electrical automation system fault detection method of claim 4 or 5, wherein, The fault detection network model comprises: a plurality of convolution layers; a plurality of pooling layers, which are arranged one by one corresponding to the convolution layers and connected at the output end of the convolution layers; and a fully connected layer connected at the output end of the last pooling layer.
7. The electrical automation system fault detection method of claim 6, wherein, Before step two, the temperature and vibration signals are cleaned to remove errors and outliers.
8. The electrical automation system fault detection method of claim 7, wherein, Before step three, it further comprises: The noise and redundant information in the audio are reduced by pre-emphasis, frame division and windowing.