Comprehensive monitoring method for abnormal working conditions of 500kV outgoing line field based on multi-modal data
Through multimodal data acquisition and deep learning model, real-time monitoring of working conditions in the 500kV outbound field is achieved, which solves the problem that the existing technology cannot accurately monitor, improves the accuracy and timeliness of fault judgment, and avoids equipment damage and Security Question.
Patent Information
- Application Number
- CN202411250275.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-06
- Publication Date
- 2025-01-10
AI Technical Summary
The prior art cannot realize online real-time monitoring of 500kV outbound working conditions on the basis of ensuring accuracy, and cannot detect potential faults in time, resulting in equipment damage or safety problems.
Using multimodal data acquisition and deep learning models, the soundprint data, vibration signals, thermal imaging, voltage and current signals are captured through bone conduction sensors, vibration sensors, infrared cameras, voltage transformers, and current transformers, and data processing and encoding are carried out. , build a multimodal weighted feature representation fusion and fault identification model based on deep learning, and realize real-time monitoring of 500kV outgoing field faults.
Real-time online monitoring of complex abnormalities in the 500kV outbound working conditions is realized, which improves the accuracy and timeliness of fault judgments and avoids equipment damage and safety problems.
Smart Images
Figure CN119298345A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of abnormal operating condition monitoring of hydropower stations, and in particular to a comprehensive monitoring method for abnormal operating condition of a 500kV outgoing line field based on multi-modal data. Background Art
[0002] The 500kV outgoing line field of a large hydropower station is a key part of the power system. It connects transformers and transmission lines and is the power supply center for the hydropower station to transmit electricity to various places. Abnormal working conditions of the 500kV outgoing line field include partial discharge of porcelain insulators, inductive discharge of lines, abnormal mechanism and other faults. If the above faults are not discovered in time and corresponding safety measures are not taken, major accidents will occur. Therefore, real-time online abnormal working condition monitoring of the 500kV outgoing line field is of great significance, which can detect potential faults early and avoid equipment damage, fire or other safety problems.
[0003] Currently, there is almost no online real-time monitoring of abnormal working conditions of 500kV outgoing line fields. Most large power plants only install voltage transformers on the outgoing lines and upload the line voltage to the monitoring system to estimate abnormal working conditions of the outgoing line fields by observing the line voltage. This monitoring method cannot meet the requirements of online real-time monitoring of complex abnormal working conditions.
[0004] Since the existing technology only monitors the voltage of the outgoing field line through a mutual inductor, when the voltage of the outgoing field line is abnormal, it cannot be directly determined that the outgoing field has an abnormal working condition, and it may also be that other equipment or lines in the power grid have a fault. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a comprehensive monitoring method for abnormal operating conditions of a 500kV outgoing line field based on multimodal data, which realizes real-time online monitoring of porcelain insulators, mechanisms, and high-voltage line faults in the 500kV outgoing line field through technologies such as voiceprint recognition and signal analysis, thereby solving the problem that the existing technology cannot perform online monitoring while ensuring accuracy, and filling the gap in the real-time online monitoring industry of 500kV outgoing line fields.
[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a comprehensive monitoring method for abnormal working conditions of a 500kV outgoing line field based on multimodal data, comprising the following steps: S1. Multimodal data acquisition: bone conduction sensors, vibration sensors, infrared cameras, voltage transformers, and current transformers are placed at appropriate locations in the 500kV outgoing line field to capture the voiceprint data, base vibration signals, outgoing line field thermal imaging, voltage and current signals of the outgoing line field respectively; S2, multi-modal data transmission, after amplifying and converting the vibration signal, voltage signal and current signal collected by the sensor, the voiceprint data, base vibration signal, outgoing line field thermal imaging, voltage and current signals are transmitted to the cloud data platform of the backend server via data cable for data analysis and fault judgment; S3, multimodal data processing, for the collected data stream, each type of data is intercepted as a single sample in the time period T=16s, that is, each sample contains 5 data types, namely voiceprint signal, vibration signal, infrared thermal image video, voltage signal, and current signal; S4, multimodal data encoding, refer to the data capture time (turbine unit start-up and shutdown time period, rainy time period, snowy time period, windy time period, no start-up and shutdown time period and clear weather time period), select representative samples from the sample library to form the training set, validation set, and test set; The non-fault samples are labeled and set to 0, and the insulator discharge samples are labeled and set to 1; A multimodal weighted feature representation fusion and fault identification model based on deep learning is constructed. The training set samples are input into the deep learning model for model training to learn the high-level encoding of the samples. Then, the validation set samples are input into the deep learning model to fine-tune the model. Finally, the test set is input into the deep learning model to realize fault classification evaluation.
[0007] S5, fault prediction, real-time collection of line field soundprint data, base vibration data, line field infrared thermal imaging data, voltage and current signals, and execution of steps S2 to S3, extraction of feature vectors of line field samples and input into a multimodal weighted feature representation fusion and fault identification model based on deep learning for 500kV line field fault monitoring.
[0008] Preferably, in step S3, the voiceprint signal is processed as follows: First, the voiceprint signal is preprocessed, including pre-emphasis, framing, and windowing, and then the Mel frequency cepstrum coefficients of the voiceprint signal are extracted; for the vibration signal, a bandpass filter is first used to filter the vibration signal, and then the wavelet transform is used to transform the vibration signal into the frequency domain; for the infrared imaging data ; Where N represents the number of frames of the video stream in the time period T = 16s, and then the pixel value difference After vectorization, it is used as the infrared thermal imaging image feature; for the voltage and current signals, the voltage and current signals are smoothed by Gaussian filter, and then the voltage / current signals are transformed into the frequency domain using Hilbert-Huang transform.
[0009] Preferably, in step S3, the Mel frequency cepstrum coefficients of the audio signal are extracted according to the following steps: Pre-emphasis: Pass the audio signal through a high-pass filter to enhance the high-frequency components and improve the signal-to-noise ratio, making the signal spectrum flat; Framing: The signal is divided into short time frames with N sampling points as an observation unit. The value of N is 256 or 512. In order to avoid excessive changes between two adjacent frames, M sampling points are overlapped between two adjacent frames when taking frames. The value of M is about 1 / 3 of N.
[0010] Windowing: Multiply each frame by a Hamming window to increase the continuity of the left and right ends of the frame; Fast Fourier Transform (FFT): Perform fast Fourier transform on each frame signal after frame division and windowing to obtain the energy distribution of each frame on the spectrum, and take the modulus square of the spectrum of the audio signal to obtain the power spectrum of the audio signal; Mel filtering: Design a set of Mel filters to convert the power spectrum into Mel spectrum; Logarithmic operation: Take the absolute value of the Mel spectrum and perform a log operation to calculate the logarithmic energy of each filter output; Discrete cosine transform: Perform discrete cosine transform on the logarithmic energy and extract 12-order cepstrum coefficients from the 2nd to the 13th order; Dynamic differential parameter extraction: The 12th order cepstrum coefficient and 1 logarithmic energy are used to obtain a total of 13 static coefficients to obtain dynamic differential parameter characteristics that can reflect the dynamic characteristics of the audio signal. The dynamic differential parameter is calculated using the following formula: ; In the formula, Indicates A first-order differential, Indicates The static coefficients, represents the order of the cepstral coefficients, =2 represents the time difference of the first-order derivative.
[0011] Substitute the result of the above formula again to get the acceleration coefficient. Finally, 39 Mel frequency cepstrum coefficients (13 MFCCs, 13 first-order differential coefficients, and 13 acceleration coefficients) can be obtained for each frame of the audio file and used as the feature representation of the audio signal.
[0012] Preferably, in step S3, wavelet transform is performed on the vibration signal to extract its frequency domain features according to the following steps: Before performing discrete wavelet transform, the vibration signal is first filtered using a bandpass filter, and then the vibration signal samples are decomposed by three layers of discrete wavelet decomposition to extract scale 3 approximate coefficients, scale 3 detail coefficients, scale 2 detail coefficients and scale 1 detail coefficients; and the feature vector representing the vibration signal sample is composed of scale 3 approximate coefficients, scale 3 detail coefficients, scale 2 detail coefficients and scale 1 detail coefficients.
[0013] Preferably, in step S3, the voltage and signal are subjected to Hilbert-Huang transform to extract their frequency domain features according to the following steps: Before performing the Hilbert-Huang transform, the voltage signal / current signal is first filtered using a bandpass filter, and then the voltage signal / current signal samples are decomposed into a set of intrinsic mode functions (IMFs) using empirical mode decomposition (EMD). Then, the Hilbert transform is applied to the IMF to obtain the analytical function; and the analytical functions are concatenated to form a feature vector that represents the voltage signal / current signal samples.
[0014] Preferably, in step S4, a multimodal weighted feature representation fusion and fault recognition model based on deep learning is constructed according to the following method: S41. Construct a deep sub-branch network for each data type to learn its deep representation. The process is expressed as follows: use represents the preprocessed multimodal data, where R represents the spatial dimension of the vector, represents the number of samples, Representation data The dimension of V=5 indicates the type of data. A deep network sub-branch is constructed for each data type to learn its deep representation: ; in, Represents a four-layer fully connected network, with nodes in each layer being , Represents the binary classification output, which is the probability distribution of the possible categories of the output of the deep network; S42. In order to achieve joint classification and maximize the contribution of each data to classification, the multimodal data classification loss adopts a weighted classification loss fusion strategy. The weighted classification loss can be expressed as: ; in, yes The classification weights, Represents the one-hot form of the sample label; represents the cross entropy loss.
[0015] The present invention provides a 500kV outgoing line field abnormality comprehensive monitoring method based on multi-modal data, which has the following beneficial effects: 1. Combined with a variety of sensors, it realizes the comprehensive monitoring of abnormal working conditions of 500kV outgoing line field based on multi-modal data (voiceprint signal, vibration signal, infrared thermal imaging, voltage, current), filling the gap in the real-time online monitoring industry of 500kV outgoing line field; 2. Online real-time monitoring of complex working conditions in the 500kV outgoing line field can detect potential faults early, avoiding equipment damage, fire or other safety issues. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 is a flow chart of the method of the present invention; Figure 2 It is a schematic diagram of the structure of the sensor of the present invention; Figure 3 It is a schematic diagram of the fusion of multi-modal weighted feature representation of the present invention; Figure 4 This is a flow chart of extracting Mel-frequency cepstrum coefficients of an audio signal according to the present invention. DETAILED DESCRIPTION
[0017] like Figure 1 As shown, a comprehensive monitoring method for abnormal working conditions of a 500kV outgoing line field based on multi-modal data includes the following steps: S1. Multimodal data acquisition: bone conduction sensors, vibration sensors, infrared cameras, voltage transformers, and current transformers are placed at appropriate locations in the 500kV outgoing line field to capture the voiceprint data, base vibration signals, outgoing line field thermal imaging, voltage and current signals of the outgoing line field respectively; Preferably, Figure 2 As shown, a bone conduction sensor 101, 102, 103 is arranged on the three-phase bases A, B, and C respectively; and a vibration sensor 104, 105, 106 is arranged on the three-phase bases A, B, and C respectively; a voltage transformer 107, 108, 109 is arranged at the three-phase outgoing line of A, B, and C respectively, and a current transformer 110, 111, 112 is arranged at the three-phase outgoing line of A, B, and C respectively; an infrared camera 113 is arranged on the main factory wall obliquely above the B phase.
[0018] The bone conduction sensor uses the ZKHY-IVOD-001 industrial diagnostic pickup with a frequency response of 20-20kHz; the vibration sensor uses the piezoelectric vibration sensor WV-100; the infrared camera uses the Hikvision DS-2CD3347G0-I camera.
[0019] S2, multi-modal data transmission, after amplifying and converting the vibration signal, voltage signal and current signal collected by the sensor, the voiceprint data, base vibration signal, outgoing line field thermal imaging, voltage and current signals are transmitted to the cloud data platform of the backend server via data cable for data analysis and fault judgment; S3, multimodal data processing, for the collected data stream, each type of data is intercepted as a single sample in the time period T=16s, that is, each sample contains 5 data types, namely voiceprint signal, vibration signal, infrared thermal image video, voltage signal, and current signal; S4, multimodal data encoding, refer to the data capture time (turbine unit start-up and shutdown time period, rainy time period, snowy time period, windy time period, no start-up and shutdown time period and clear weather time period), select representative samples from the sample library to form the training set, validation set, and test set; The non-fault samples are labeled and set to 0, and the insulator discharge samples are labeled and set to 1; Construct a multimodal weighted feature representation fusion and fault recognition model based on deep learning, such as Figure 3 As shown in the figure, the training set samples are input into the deep learning model for model training to learn the high-level encoding of the samples; then the validation set samples are input into the deep learning model to fine-tune the model; finally, the test set is input into the deep learning model to realize fault classification evaluation.
[0020] S5, fault prediction, real-time collection of line field soundprint data, base vibration data, line field infrared thermal imaging data, voltage and current signals, and execution of steps S2 to S3, extraction of feature vectors of line field samples and input into a multimodal weighted feature representation fusion and fault identification model based on deep learning for 500kV line field fault monitoring.
[0021] Preferably, in step S3, the voiceprint signal is processed as follows: First, the voiceprint signal is preprocessed, including pre-emphasis, framing, and windowing, and then the Mel frequency cepstrum coefficients of the voiceprint signal are extracted; for the vibration signal, a bandpass filter is first used to filter the vibration signal, and then the wavelet transform is used to transform the vibration signal into the frequency domain; for the infrared imaging data ; Where N represents the number of frames of the video stream in the time period T = 16s, and then the pixel value difference After vectorization, it is used as the infrared thermal imaging image feature; for the voltage and current signals, the voltage and current signals are smoothed by Gaussian filter, and then the voltage / current signals are transformed into the frequency domain using Hilbert-Huang transform.
[0022] Before extracting dynamic features, the signal is first preprocessed (pre-emphasis, framing, windowing), fast Fourier transform, Mei filter bank, logarithmic operation, discrete cosine transform, such as Figure 4 shown.
[0023] Preferably, in step S3, the Mel frequency cepstrum coefficients of the audio signal are extracted according to the following steps: Pre-emphasis: Pass the audio signal through a high-pass filter to enhance the high-frequency components and improve the signal-to-noise ratio, making the signal spectrum flat; Framing: The signal is divided into short time frames with N sampling points as an observation unit. The value of N is 256 or 512. In order to avoid excessive changes between two adjacent frames, M sampling points are overlapped between two adjacent frames when taking frames. The value of M is about 1 / 3 of N.
[0024] Windowing: Multiply each frame by a Hamming window to increase the continuity of the left and right ends of the frame; Fast Fourier Transform (FFT): Perform fast Fourier transform on each frame signal after frame division and windowing to obtain the energy distribution of each frame on the spectrum, and take the modulus square of the spectrum of the audio signal to obtain the power spectrum of the audio signal; Mel filtering: Design a set of Mel filters to convert the power spectrum into Mel spectrum; Logarithmic operation: Take the absolute value of the Mel spectrum and perform a log operation to calculate the logarithmic energy of each filter output; Discrete cosine transform: Perform discrete cosine transform on the logarithmic energy and extract 12-order cepstrum coefficients from the 2nd to the 13th order; Dynamic differential parameter extraction: The 12th order cepstrum coefficient and 1 logarithmic energy are used to obtain a total of 13 static coefficients to obtain dynamic differential parameter characteristics that can reflect the dynamic characteristics of the audio signal. The dynamic differential parameter is calculated using the following formula: ; In the formula, Indicates A first-order differential, Indicates The static coefficients, represents the order of the cepstral coefficients, =2 represents the time difference of the first-order derivative.
[0025] Substitute the result of the above formula again to get the acceleration coefficient. Finally, 39 Mel frequency cepstrum coefficients (13 MFCCs, 13 first-order differential coefficients, and 13 acceleration coefficients) can be obtained for each frame of the audio file and used as the feature representation of the audio signal.
[0026] Preferably, in step S3, wavelet transform is performed on the vibration signal to extract its frequency domain features according to the following steps: Before performing discrete wavelet transform, the vibration signal is first filtered using a bandpass filter, and then the vibration signal samples are decomposed by three layers of discrete wavelet decomposition to extract scale 3 approximate coefficients, scale 3 detail coefficients, scale 2 detail coefficients and scale 1 detail coefficients; and the feature vector representing the vibration signal sample is composed of scale 3 approximate coefficients, scale 3 detail coefficients, scale 2 detail coefficients and scale 1 detail coefficients.
[0027] Preferably, in step S3, the voltage and signal are subjected to Hilbert-Huang transform to extract their frequency domain features according to the following steps: Before performing the Hilbert-Huang transform, the voltage signal / current signal is first filtered using a bandpass filter, and then the voltage signal / current signal samples are decomposed into a set of intrinsic mode functions (IMFs) using empirical mode decomposition (EMD). Then, the Hilbert transform is applied to the IMF to obtain the analytical function; and the analytical functions are concatenated to form a feature vector that represents the voltage signal / current signal samples.
[0028] Preferably, in step S4, a multimodal weighted feature representation fusion and fault recognition model based on deep learning is constructed according to the following method: S41. Construct a deep sub-branch network for each data type to learn its deep representation. The process is expressed as follows: use represents the preprocessed multimodal data, where R represents the spatial dimension of the vector, represents the number of samples, Representation data The dimension of V=5 indicates the type of data. A deep network sub-branch is constructed for each data type to learn its deep representation: ; in, Represents a four-layer fully connected network, with nodes in each layer being , Represents the binary classification output, which is the probability distribution of the possible categories of the output of the deep network; S42. In order to achieve joint classification and maximize the contribution of each data to classification, the multimodal data classification loss adopts a weighted classification loss fusion strategy. The weighted classification loss can be expressed as: ; in, yes The classification weights, Represents the one-hot form of the sample label; represents the cross entropy loss.
[0029] The present invention includes multimodal data acquisition, multimodal data transmission, multimodal data processing, multimodal data comprehensive coding, and fault prediction. Multimodal data acquisition is mainly completed by sensor hardware to realize online real-time data acquisition; multimodal data transmission mainly transmits the collected data to the server; multimodal data processing is mainly used to pre-process the multimodal data; comprehensive fault diagnosis is based on the multimodal real-time data collected by the sensor, and the fault diagnosis model is constructed to comprehensively monitor the working conditions of the outgoing line field to determine whether there are known faults or unknown abnormalities in the 500kV outgoing line field.
[0030] By installing bone conduction microphones, vibration sensors, cameras with thermal infrared temperature measurement function, voltage transformers, and current transformers in appropriate areas, and integrating multimodal data into the same system, a comprehensive analysis of the 500kV outgoing line field conditions can be achieved.
[0031] Through deep learning of various data parameters and logical design of fault judgment process, comprehensive diagnosis of multi-modal abnormal operating conditions of 500kV outgoing line field is realized, which greatly improves the real-time and accuracy of abnormal operating conditions or fault judgment of 500kV outgoing line field.
[0032] The above embodiments are only preferred technical solutions of the present invention and should not be regarded as limiting the present invention. The protection scope of the present invention shall be the technical solutions recorded in the claims, including equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.
Claims
1. A comprehensive monitoring method for abnormal working conditions of 500kV outgoing line field based on multimodal data, characterized in that: The following steps are involved: S1. Multimodal data acquisition: bone conduction sensors, vibration sensors, infrared cameras, voltage transformers, and current transformers are placed at appropriate locations in the 500kV outgoing line field to capture the voiceprint data, base vibration signals, outgoing line field thermal imaging, voltage and current signals of the outgoing line field respectively; S2, multi-modal data transmission, after amplifying and converting the vibration signal, voltage signal and current signal collected by the sensor, the voiceprint data, base vibration signal, outgoing line field thermal imaging, voltage and current signals are transmitted to the cloud data platform of the backend server via data cable for data analysis and fault judgment; S3, multimodal data processing, for the collected data stream, each type of data is intercepted as a single sample in the time period T=16s, that is, each sample contains 5 data types, namely voiceprint signal, vibration signal, infrared thermal image video, voltage signal, and current signal; S4, multimodal data encoding, refer to the data capture time, select representative samples from the sample library to form the training set, validation set, and test set; The non-fault samples are labeled as 0, and the fault samples are labeled as 1; Construct a multimodal weighted feature representation fusion and fault identification model based on deep learning. Input the training set samples into the deep learning model for model training and learn the high-level encoding of the samples. Then input the validation set samples into the deep learning model to fine-tune the model. Finally, input the test set into the deep learning model to implement fault classification evaluation. S5, fault prediction, real-time collection of line field soundprint data, base vibration data, line field infrared thermal imaging data, voltage and current signals, and execution of steps S2 to S3, extraction of feature vectors of line field samples and input into a multimodal weighted feature representation fusion and fault identification model based on deep learning for 500kV line field fault monitoring.
2. According to claim 1, a 500kV outgoing line field abnormality comprehensive monitoring method based on multi-modal data is characterized in that: In step S3, the voiceprint signal is processed as follows: First, the voiceprint signal is preprocessed, including pre-emphasis, framing, and windowing, and then the Mel frequency cepstrum coefficients of the voiceprint signal are extracted; for the vibration signal, a bandpass filter is first used to filter the vibration signal, and then the wavelet transform is used to transform the vibration signal into the frequency domain; for the infrared imaging data ; Where N represents the number of video frames in the video stream within the time period T = 16s, and then the pixel value difference After vectorization, it is used as the infrared thermal imaging image feature; for the voltage and current signals, the voltage and current signals are smoothed by Gaussian filter, and then the voltage / current signals are transformed into the frequency domain using Hilbert-Huang transform.
3. According to claim 2, a 500kV outgoing line field abnormality comprehensive monitoring method based on multi-modal data is characterized in that: In step S3, the Mel-frequency cepstrum coefficients of the audio signal are extracted according to the following steps: Pre-emphasis: Pass the audio signal through a high-pass filter to enhance the high-frequency components and improve the signal-to-noise ratio, making the signal spectrum flat; Framing: Take N sampling points as an observation unit and divide the signal into short time frames; N is 256 or 512; in order to avoid too large changes between two adjacent frames, M sampling points are overlapped between two adjacent frames when taking frames, and the value of M is about 1 / 3 of N; Windowing: Multiply each frame by a Hamming window to increase the continuity of the left and right ends of the frame; Fast Fourier Transform (FFT): Perform fast Fourier transform on each frame signal after frame division and windowing to obtain the energy distribution of each frame on the spectrum, and take the modulus square of the spectrum of the audio signal to obtain the power spectrum of the audio signal; Mel filtering: Design a set of Mel filters to convert the power spectrum into Mel spectrum; Logarithmic operation: Take the absolute value of the Mel spectrum and perform a log operation to calculate the logarithmic energy of each filter output; Discrete cosine transform: Perform discrete cosine transform on the logarithmic energy and extract 12-order cepstrum coefficients from the 2nd to the 13th order; Dynamic differential parameter extraction: The 12th-order cepstrum coefficients and 1 logarithmic energy, a total of 13 static coefficients, are used to obtain dynamic differential parameter characteristics that can reflect the dynamic characteristics of the audio signal; the dynamic differential parameters are calculated using the following formula: ； In the formula, Indicates A first-order differential, Indicates The static coefficients, represents the order of the cepstral coefficients, =2 represents the time difference of the first-order derivative; Substitute the result of the above formula again to get the acceleration coefficient; finally, 39 Mel frequency cepstrum coefficients can be obtained for each frame of the audio file and used as the feature representation of the audio signal.
4. According to claim 2, a 500kV outgoing line field abnormality comprehensive monitoring method based on multimodal data is characterized in that: In step S3, wavelet transform is performed on the vibration signal to extract its frequency domain features according to the following steps: Before performing discrete wavelet transform, the vibration signal is first filtered using a bandpass filter, and then the vibration signal samples are decomposed by three layers of discrete wavelet decomposition to extract scale 3 approximate coefficients, scale 3 detail coefficients, scale 2 detail coefficients and scale 1 detail coefficients; and the feature vector representing the vibration signal sample is composed of scale 3 approximate coefficients, scale 3 detail coefficients, scale 2 detail coefficients and scale 1 detail coefficients.
5. According to claim 2, a 500kV outgoing line field abnormality comprehensive monitoring method based on multi-modal data is characterized in that: In step S3, the voltage and signal are subjected to Hilbert-Huang transform to extract their frequency domain features according to the following steps: Before performing the Hilbert-Huang transform, the voltage signal / current signal is first filtered using a bandpass filter, and then the voltage signal / current signal samples are decomposed into a set of intrinsic mode functions using empirical mode decomposition. Then, the Hilbert transform is applied to the IMF to obtain the analytical function; and the analytical functions are concatenated to form a feature vector that represents the voltage signal / current signal samples.
6. According to claim 1, a 500kV outgoing line field abnormality comprehensive monitoring method based on multi-modal data is characterized in that: In step S4, a multimodal weighted feature representation fusion and fault recognition model based on deep learning is constructed according to the following method: S41. Build a deep sub-branch network for each data type to learn its deep representation. The process is as follows: use represents the preprocessed multimodal data, where R represents the spatial dimension of the vector, represents the number of samples, Representation data The dimension of V=5 indicates the type of data. A deep network sub-branch is constructed for each data type to learn its deep representation: ； in, Represents a four-layer fully connected network, with nodes in each layer being , Represents the binary classification output, which is the probability distribution of the possible categories of the output of the deep network; S42. In order to achieve joint classification and maximize the contribution of each data to classification, the multimodal data classification loss adopts a weighted classification loss fusion strategy. The weighted classification loss can be expressed as: ； in, yes The classification weights, Represents the one-hot form of sample labels; represents the cross entropy loss.