Power equipment fault prediction method based on multi-modal data and related equipment

Abstract

The invention discloses a power equipment fault prediction method based on multi-modal data and related equipment, and belongs to the technical field of intelligent monitoring and fault prediction of power equipment, and the method comprises the steps: collecting and preprocessing the multi-modal operation data of the power equipment; extracting features of the preprocessed multi-modal operation data and performing global feature fusion to obtain multi-modal features; inputting the multi-modal features into a pre-trained fault prediction model to obtain a fault category and an occurrence probability of the power equipment; the fault prediction model is obtained by inputting the multi-modal features into a hybrid neural network for training. Multi-modal operation data of power equipment is collected, key features are extracted by using a deep learning model, multi-modal data fusion is performed, and a fault prediction model is constructed based on a hybrid neural network. Real-time fault early warning and maintenance suggestions are provided, the operation reliability of power equipment is improved, the unplanned shutdown risk is reduced, and the method is suitable for health management of transformer substations, high-voltage power transmission equipment and wind generating sets.

Description

Technical Field

[0001] The present invention belongs to the technical field, and specifically relates to a power equipment fault prediction method based on multi-modal data and related equipment. Background Art

[0002] During the long-term operation of power equipment, affected by various factors such as environmental factors, grid load changes, and mechanical wear, various faults may occur, such as insulation aging, component wear, overload operation, and partial discharge. These faults not only affect the normal operation of the equipment but also pose a threat to the stability of the entire power grid. Therefore, real-time monitoring of the operating state of power equipment and predicting potential faults through effective technical means are of great significance for improving the safety and reliability of power grid operation. With the development of intelligent monitoring technology, the multi-modal data fusion method has gradually become an important research direction for power equipment state assessment. This method can comprehensively utilize information such as electrical signals, mechanical vibrations, and thermal imaging data collected by different sensors, providing a more comprehensive analysis basis for power equipment fault prediction.

[0003] Currently, the fault monitoring and prediction methods for power equipment mainly include methods based on physical modeling, methods based on statistical analysis, and intelligent algorithms based on data-driven. Among them, physical modeling methods usually rely on expert experience to establish a mathematical model of the equipment and conduct analysis. However, due to the complex operating environment of power equipment, it is difficult for the model to comprehensively cover the actual working conditions, resulting in limited prediction accuracy. The method based on statistical analysis can identify the change trend of the equipment operating state by analyzing the trend and abnormal detection of the equipment operation historical data, but it is difficult to accurately predict specific fault types. In recent years, data-driven intelligent prediction methods have developed rapidly. Prediction models based on machine learning and deep learning can be trained through large-scale equipment operation data and have high prediction accuracy. However, most of the existing intelligent prediction methods rely on single-modal data, such as only using electrical signals or vibration signals for analysis, and it is difficult to make full use of the advantages of multi-modal data. In addition, the existing methods have insufficient data processing and model optimization capabilities during the fault prediction process and are easily affected by factors such as data noise and sensor errors, resulting in unstable prediction results.

[0004] Currently, the fault prediction for power equipment still faces the problem of insufficient multi-modal data fusion, making it difficult to effectively utilize the comprehensive information of various data such as electrical signals, vibration signals, and thermal imaging, resulting in limited prediction accuracy and generalization ability. In addition, due to the differences in sampling frequencies and formats of different modal data, how to efficiently preprocess multi-modal data, extract effective features, and combine time series modeling during the prediction process to achieve accurate fault judgment is an urgent problem to be solved in the current technology. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a power equipment fault prediction method and related equipment based on multi-modal data in view of the deficiencies in the above-mentioned prior art, so as to solve the technical problems of how to efficiently preprocess multi-modal data, extract effective features, and combine time series modeling in the prediction process to achieve accurate fault judgment.

[0006] The present invention adopts the following technical solutions: In the first aspect, the present invention provides a power equipment fault prediction method based on multi-modal data, including the following steps: Collect multi-modal operation data of power equipment and preprocess it; Extract the features of the preprocessed multi-modal operation data and perform global feature fusion to obtain multi-modal features; Input the multi-modal features into a pre-trained fault prediction model to obtain the fault category and occurrence probability of the power equipment; The fault prediction model is obtained by training by inputting multi-modal features into a hybrid neural network.

[0007] Preferably, the multi-modal operation data includes: electrical signals, mechanical vibrations, thermal imaging, sound data, and environmental data; The preprocessing includes: signal denoising, time series alignment, and data format standardization; Signal denoising uses short-time Fourier transform or wavelet transform to remove high-frequency noise in the vibration signal; time series alignment uses an interpolation method to keep data with different sampling frequencies synchronized on the time axis.

[0008] Preferably, Transformer is used to perform global feature fusion on the features of the preprocessed multi-modal operation data; The features of the preprocessed multi-modal operation data include: local features of vibration signals and thermal imaging data, and time series features of electrical signals and environmental data; The local features of vibration signals and thermal imaging data are extracted by using a convolutional neural network; the time series features of electrical signals and environmental data are modeled by using a long short-term memory network; Transformer adopts a self-attention mechanism to establish a global dependence relationship among multi-modal operation data.

[0009] Further preferably, the fault prediction model realizes in-depth correlation analysis of multi-modal features by combining a convolutional neural network, a long short-term memory network, and Transformer, and adopts an optimization strategy to train the fault prediction model; The optimization strategy includes loss function optimization, Bayesian optimization, and reinforcement learning optimization; During the training process, a data augmentation strategy is adopted; the data augmentation strategy includes time series data augmentation and image data augmentation.

[0010] In a second aspect, the present invention provides a power equipment fault prediction system based on multi-modal data, including: A data acquisition and preprocessing module for acquiring and preprocessing the multi-modal operation data of power equipment; A feature extraction and fusion module for extracting the features of the preprocessed multi-modal operation data and performing global feature fusion to obtain multi-modal features; A fault prediction module for inputting the multi-modal features into a pre-trained fault prediction model based on a hybrid neural network to obtain the fault category and occurrence probability of the power equipment.

[0011] Preferably, the data acquisition and preprocessing module includes multiple sensors, and the sensors transmit the multi-modal operation data of the power equipment in a wireless or wired manner to achieve remote data monitoring; The fault prediction module includes: a convolutional neural network, a long short-term memory network, and a Transformer; the convolutional neural network is responsible for extracting local features of vibration signals and thermal imaging data; the long short-term memory network is responsible for time series modeling of electrical signals and environmental data; the Transformer is responsible for global feature fusion.

[0012] Preferably, it further includes an intelligent optimization module and a fault warning and visualization module; The intelligent optimization module adopts a reinforcement learning mechanism to dynamically adjust the learning strategy of the fault prediction model according to the prediction error, and improve the prediction accuracy of the fault prediction model; The fault warning and visualization module displays the health status, prediction results, and maintenance suggestions of the equipment based on the power operation and maintenance platform, and supports the visual analysis of historical fault data.

[0013] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, including a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned power equipment fault prediction method based on multi-modal data are implemented.

[0014] In a fourth aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the above-mentioned power equipment fault prediction method based on multi-modal data are implemented.

[0015] In a fifth aspect, the present invention provides an electronic device, including a computer program, and when the computer program is executed by the electronic device, the steps of the above-mentioned power equipment fault prediction method based on multi-modal data are implemented.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: Through multi-modal data fusion, the present invention improves the accuracy and stability of power equipment fault prediction. By comprehensively analyzing electrical signals, vibrations, thermal images, sounds, and environmental data, fault detection becomes more comprehensive, avoiding the limitations caused by a single data source. By using deep learning methods to extract key features, combined with convolutional neural networks (CNNs) for local pattern recognition, long short-term memory networks (LSTMs) for time series modeling, and Transformers for global feature fusion, the prediction model can simultaneously focus on the instantaneous state and long-term trends of the equipment, improving the ability to evaluate the health status of the equipment under complex working conditions. In addition, through Bayesian optimization and reinforcement learning techniques, the model parameters are adaptively adjusted, enhancing the generalization ability and noise resistance of the model, enabling it to maintain a high prediction accuracy under different types of power equipment and environmental conditions. The fault prediction system of the present invention can provide real-time warnings and maintenance suggestions, improve the reliability of power equipment operation, reduce the risk of unplanned outages, and provide a scientific basis for power system operation and maintenance.

[0017] The fault prediction system disclosed by the present invention can input multi-modal operation data of power equipment in real time and output the equipment fault category and its occurrence probability based on the trained fault prediction model. When the prediction probability exceeds the set threshold, the system can automatically trigger a fault warning and provide corresponding maintenance suggestions, helping operation and maintenance personnel take timely measures to prevent the occurrence or expansion of faults. By predicting equipment faults in advance, operation and maintenance personnel can formulate more reasonable maintenance plans and avoid unplanned outages caused by sudden faults. The visualization interface and detailed report function of the system enable operation and maintenance personnel to intuitively understand the equipment status and improve decision-making efficiency. The present invention is applicable to different types of power equipment, such as substation equipment, wind turbines, high-voltage transmission lines, etc., and has wide applicability. Through continuous learning and optimization, the system can adapt to changes in the equipment operation state and improve the ability to identify abnormal situations. Through multi-modal data fusion and deep learning techniques, the present invention significantly improves the accuracy and lead time of power equipment fault prediction, optimizes the operation and maintenance strategy, reduces the risk of sudden faults, and provides a strong guarantee for the safety and stability of the power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of a power equipment fault prediction method based on multi-modal data according to Embodiment 1 of the present invention; Figure 2 It is a schematic diagram of the modules of a power equipment fault prediction system based on multi-modal data according to Embodiment 2 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0020] In the description of the present invention, it should be understood that the terms "include" and "comprise" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0021] It should also be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0022] It should be further understood that the term "and / or" used in the specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in the present invention generally represents an "or" relationship between the contextually related objects.

[0023] It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present invention to describe preset ranges, etc., these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range can also be referred to as the second preset range, and similarly, the second preset range can also be referred to as the first preset range.

[0024] Depending on the context, the word "if" as used herein can be interpreted as "when" or "while" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if determined" or "if detecting (stated condition or event)" can be interpreted as "when determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)".

[0025] The present invention provides a power equipment fault prediction method based on multimodal data, including the following steps: Collect multi-modal operation data of power equipment, where the multi-modal operation data includes electrical signals, mechanical vibrations, thermal imaging, sound data, and environmental data; Preprocess the multi-modal operation data, including signal denoising, time series alignment, and data format standardization, to ensure the fusion consistency of different modal data; Extract multi-modal data features, where convolutional neural network (CNN) is used to extract local features of vibration signals and thermal imaging data, long short-term memory network (LSTM) is used to model the time series features of electrical signals and environmental data, and Transformer is used for global feature fusion; Construct a fault prediction model based on a hybrid neural network (HNN), which realizes in-depth correlation analysis of multi-modal operation features by combining CNN, LSTM, and Transformer; Adopt an optimization strategy to train the fault prediction model, including loss function optimization, Bayesian optimization, and reinforcement learning optimization, to improve the prediction accuracy and generalization ability of the model; During the operation of the equipment, input the multi-modal operation data of the power equipment collected in real time, and based on the trained fault prediction model, output the equipment fault category and its occurrence probability. When the prediction probability exceeds the set threshold, trigger a fault warning and provide corresponding maintenance suggestions.

[0026] Signal denoising uses short-time Fourier transform or wavelet transform to remove high-frequency noise in vibration signals and improve data quality.

[0027] Time series alignment uses an interpolation method to keep data with different sampling frequencies synchronized on the time axis to ensure data consistency.

[0028] Transformer uses a self-attention mechanism to establish global dependencies between multi-modal data and improve the comprehensive analysis ability of fault prediction.

[0029] During the training process of the fault prediction model, a data augmentation strategy is adopted, including time series data augmentation and image data augmentation, to improve the adaptability of the model to different equipment working conditions.

[0030] The present invention also provides a power equipment fault prediction system based on multi-modal data, including: A data acquisition module for collecting multi-modal data, including electrical signals, mechanical vibrations, thermal imaging, sound data, and environmental data, and transmitting it to the data processing center; A data preprocessing module, connected to the data acquisition module, for denoising, time series alignment, and format standardization of multimodal data; A feature extraction and fusion module, connected to the data preprocessing module, for extracting key features of multimodal data and fusing multimodal features through a deep learning model to obtain multimodal features; A fault prediction module, connected to the feature extraction and fusion module, constructs a fault prediction model based on a hybrid neural network (HNN) to calculate the fault category and its occurrence probability; An intelligent optimization module, connected to the fault prediction module, uses Bayesian optimization and reinforcement learning to optimize the fault prediction model and improve the adaptability of the model; A fault warning and visualization module, connected to the fault prediction module, for receiving prediction results and visually displaying device status and prediction information on the power operation and maintenance platform.

[0031] The data acquisition module includes multiple sensors, and the sensors transmit data wirelessly or wired to achieve remote data monitoring.

[0032] The fault prediction module uses a hybrid neural network, where CNN is responsible for extracting local features of vibration signals and thermal imaging data, LSTM is responsible for time series modeling of electrical signals and environmental data, and Transformer is responsible for global feature fusion.

[0033] The intelligent optimization module adopts a reinforcement learning mechanism to dynamically adjust the learning strategy of the fault prediction model according to the prediction error and improve the prediction accuracy of the model.

[0034] The fault warning and visualization module displays the health status, prediction results, and maintenance suggestions of the device based on the power operation and maintenance platform and supports visual analysis of historical fault data.

[0035] In another embodiment of the present invention, a terminal device is provided. The terminal device includes a processor and a memory. The memory is used to store a computer program, and the computer program includes program instructions. The processor is used to execute the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function. The processor described in the embodiment of the present invention can be used for the operation of the power equipment fault prediction method based on multimodal data.

[0036] In another embodiment of the present invention, a storage medium is also provided, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in the terminal device and is used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and, of course, the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, and the operating system of the terminal is stored in this storage space. And, one or more instructions suitable for being loaded and executed by the processor are also stored in this storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory.

[0037] One or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the power equipment fault prediction method based on multimodal data in the above embodiments.

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components described and shown in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0039] Embodiment 1 This embodiment provides a power equipment fault prediction method based on multi-modal data. This method makes full use of multi-modal data such as vibration, thermal imaging, electrical signals, sound, and environmental parameters, and combines a deep learning model for feature extraction, pattern recognition, and trend analysis to achieve intelligent fault detection and accurate prediction of power equipment.

[0040] See Figure 1 It is a flowchart of a power equipment fault prediction method based on multi-modal data according to Embodiment 1 of the present invention; as can be seen from the figure, this method includes: data acquisition, data preprocessing, feature extraction, fault prediction model construction, model optimization and training, and prediction and warning mechanisms to ensure early detection of power equipment anomalies and improve operation and maintenance efficiency.

[0041] During the data acquisition process, multiple sensors are used to monitor the operating state of power equipment, and obtain electrical signals (such as current, voltage, power factor, and harmonics), mechanical vibrations (such as bearing vibration and rotor imbalance), thermal imaging data (temperature distribution on the surface of the equipment), sound data (acoustic characteristics of equipment operation), and environmental data (such as temperature, humidity, and atmospheric pressure). These data can comprehensively reflect the equipment operating state and provide high-quality input for subsequent fault analysis.

[0042] Data preprocessing is an important step to ensure data quality. For different types of data, different preprocessing methods are adopted. For example, vibration signals are first subjected to short-time Fourier transform (STFT) or wavelet transform to extract time-frequency features and remove noise interference at the same time. The sampling frequencies of time series data are different, so interpolation methods are used for alignment to ensure the time synchronization of the data. Thermal imaging data removes irrelevant information through background segmentation and only retains the temperature distribution data of key equipment parts. All data are standardized in format during the preprocessing stage for subsequent feature extraction and fusion.

[0043] Feature extraction uses different deep learning models to adapt to the characteristics of multi-modal data. Convolutional Neural Network (CNN) is used for local feature extraction of vibration signals and thermal imaging data. CNN can automatically learn the key features in signals and images through multiple convolutional layers, such as abnormal peaks in vibration patterns or hot spot areas in thermal imaging data. Long Short-Term Memory Network (LSTM) is used for time series modeling of electrical signals and environmental data. This model captures time dependencies through its memory mechanism to predict the future operating trends of the device. The Transformer structure is responsible for global feature fusion of multi-modal data. Its self-attention mechanism can capture the correlations between different modal data. For example, when the temperature of the device rises and the vibration amplitude increases, this model can automatically identify the potential relationships between these features and give a high-confidence anomaly warning.

[0044] The fault prediction model is built based on a Hybrid Neural Network (HNN). This model integrates the advantages of CNN, LSTM, and Transformer to form a multi-level feature learning architecture. In this model, CNN is responsible for extracting local spatial features, such as vibration patterns of bearing wear and rotor imbalance, and the distribution of hot spots on the device; LSTM is responsible for learning the historical operating trends of electrical equipment, such as the impact of long-term high load on the device; Transformer is responsible for global modeling of multi-modal features, enabling efficient fusion of information between different data sources and improving the accuracy of prediction.

[0045] The model is trained using a supervised learning method, based on a large amount of historical operating data and labeled fault data. During the training process, the cross-entropy loss function is used to calculate the error between the predicted fault category of the model and the true fault category, and the network parameters are updated through the backpropagation algorithm. To improve the robustness of the model, the Bayesian optimization algorithm is used to automatically adjust hyperparameters, such as the learning rate, hidden layer size, and batch size, so that the model has good generalization ability under different device types and operating conditions. In addition, a reinforcement learning mechanism is introduced into the optimization process, enabling the model to dynamically adjust the learning strategy according to the prediction error, thereby gradually optimizing the prediction accuracy during the training process.

[0046] In the prediction stage, when new data is input into the model, the system can calculate the probability values of each fault category. If the predicted probability of a certain fault category exceeds the set threshold, the system will automatically trigger the warning mechanism and provide detailed fault types, occurrence probabilities, and recommended maintenance measures to the operation and maintenance personnel. For example, if it is predicted that the temperature of a certain transformer winding rises abnormally and high-frequency components appear in the vibration data, the system will infer that winding aging may occur and recommend that the operation and maintenance personnel check the winding temperature and oil quality to prevent further damage to the device.

[0047] This method significantly improves the accuracy and lead time of power equipment fault prediction through multi-modal data fusion and deep learning techniques. At the same time, it optimizes the operation and maintenance strategies and reduces the occurrence of sudden faults. This method can be widely applied to multiple power system scenarios such as substation equipment monitoring, wind turbine fault diagnosis, and hidden danger detection of high-voltage transmission lines, improving the safety and stability of the power system.

[0048] The method described in this embodiment can adapt to different types of power equipment and, combined with technologies such as reinforcement learning and Bayesian optimization, continuously improve the prediction accuracy, providing strong support for the health management of power equipment.

[0049] Embodiment 2 This embodiment provides a power equipment fault prediction system based on multi-modal data. The system realizes precise monitoring and intelligent early warning of power equipment through multi-modal data acquisition, intelligent data processing, deep learning feature extraction, fault prediction model construction and optimization. The core functions of the system include data acquisition, data processing, fault analysis, prediction model training and optimization, early warning mechanism and decision support to ensure the safe and stable operation of power equipment.

[0050] See Figure 2 It is a module schematic diagram of a power equipment fault prediction system based on multi-modal data according to Embodiment 2 of the present invention; it can be seen from the figure that the system includes a data acquisition module, a data preprocessing module, a feature extraction and fusion module, a fault prediction module, an intelligent optimization module, and a fault early warning and visualization module. Each module cooperates with each other to realize precise monitoring and analysis of the state of power equipment.

[0051] The data acquisition module is used to collect multi-modal data such as electrical signals, mechanical vibrations, thermal imaging, sounds, and environmental data, and transmit them to the data processing center by wireless or wired means. The data preprocessing module performs noise reduction, time alignment, and format standardization on the multi-modal data to ensure the consistency and high-quality input of the data. The feature extraction and fusion module extracts the features of different modal data through deep learning methods. For example, it uses CNN to extract vibration signal and thermal imaging features, uses LSTM to model time series features, and uses Transformer to construct global feature fusion to obtain complete equipment state information.

[0052] The fault prediction module is modeled based on a Hybrid Neural Network (HNN) and combines the trained data model to evaluate the operating status of power equipment in real-time and calculate the likelihood of potential faults. When the prediction result exceeds the set threshold, the intelligent optimization module will use reinforcement learning and Bayesian optimization techniques to dynamically adjust the model parameters to improve the prediction accuracy. Finally, the fault warning and visualization module will visualize the prediction results and provide decision-making support on the power operation and maintenance platform, including risk assessment, fault category judgment, and maintenance recommendations.

[0053] The advantage of this system lies in its ability to integrate information from different data sources and use deep learning and intelligent optimization methods for multi-modal data fusion to improve the accuracy of fault prediction. In addition, the system has good generalization ability, can be applied to different types of power equipment, and can improve the recognition ability of abnormal situations through continuous learning and optimization. Through the implementation of this system, power operation and maintenance personnel can anticipate equipment faults in advance, reduce the risk of unplanned outages, and improve the safety and stability of power grid operation.

[0054] Application Example This application example describes the application scenario of the present invention in the actual operation and maintenance of power equipment, including the complete processes such as system deployment, data collection, fault prediction, and warning response. Taking a large substation as an example, the main transformers, high-voltage circuit breakers, and transmission line equipment in this substation have been operating at high load for a long time, with potential fault risks such as equipment overload, partial discharge, and insulation aging. Therefore, this substation introduces the power equipment fault prediction system based on multi-modal data of the present invention to improve the equipment health management ability and reduce the risk of unplanned outages.

[0055] In the environment of this substation, multiple sensors are deployed at key equipment locations to collect real-time operation data. The installed electrical sensors are responsible for monitoring electrical parameters such as current, voltage, power factor, and harmonics of the main transformer and high-voltage circuit breaker; vibration sensors detect the vibration conditions of the internal windings, iron cores, cooling fans, and contacts of the high-voltage circuit breaker of the transformer; infrared thermal imaging sensors monitor the temperature distribution of the transformer, busbars, and switchgear in real-time to identify abnormal hot spots; acoustic sensors collect the acoustic wave characteristics during equipment operation to analyze partial discharge or mechanical fault signals; in addition, environmental sensors provide external environmental parameters such as temperature, humidity, and air pressure to evaluate the impact of environmental factors on equipment operation.

[0056] After data acquisition, the data is transmitted to the fault prediction system of the present invention via a wireless or fiber optic network and enters the data preprocessing stage. Since the data formats and sampling frequencies of different sensors are different, the system first standardizes the data formats and synchronizes the time to ensure the unity of different modality data. Subsequently, different preprocessing methods are adopted for different types of data. For example, wavelet transform is used to remove the high-frequency noise of vibration signals, image enhancement processing is used to optimize the contrast of thermal imaging data, and time series interpolation methods are used to align electrical signals.

[0057] In the feature extraction stage, the system uses deep learning models to extract multi-modal data features. The convolutional neural network (CNN) analyzes the temperature gradient of thermal imaging data, identifies the hot spot areas on the device surface, and combines with the vibration signal features to evaluate the mechanical state of the device. The long short-term memory network (LSTM) models the time series changes of electrical signals to detect abnormal fluctuation trends of current and voltage. The Transformer structure further fuses different modality data and uses the self-attention mechanism to calculate the correlation between data. For example, when the temperature of a transformer rises abnormally, the system can automatically associate the vibration data and electrical signals of the device to evaluate whether there are problems such as insulation aging or overload.

[0058] The fault prediction module performs inference based on the hybrid neural network (HNN) and calculates the possible fault categories and their occurrence probabilities. For example, during the operation of a substation, the thermal imaging data of a main transformer shows abnormal temperature rise, and at the same time, the vibration sensor detects high-frequency vibration signals in the winding area, while the analysis of the current data shows that the load of the transformer has been continuously at a peak value. After model calculation, the system speculates that there may be a risk of winding insulation aging in the transformer and predicts the occurrence probability of this risk to be 85%. Since this risk probability exceeds the set warning threshold of 75%, the system immediately triggers a fault warning and sends an alarm message to the substation operation and maintenance center.

[0059] After the warning mechanism is activated, the system generates a detailed fault report, including the current operating state, abnormal indicators, fault category prediction results, risk probability, and recommended maintenance measures of the transformer. For example, the report may contain "abnormal temperature rise of the winding (exceeding 85°C), abnormal vibration characteristics, continuous current load higher than 90%, predicted probability of winding insulation aging is 85%, it is recommended to monitor the winding temperature, perform insulation withstand voltage test, and appropriately reduce the load operation." The operation and maintenance personnel can view this report through the visualization interface of the operation and maintenance platform and formulate a maintenance plan according to the maintenance suggestions provided by the system.

[0060] In addition, to optimize the system's prediction ability, the intelligent optimization module dynamically adjusts the model parameters according to the accuracy of fault prediction. The system adopts reinforcement learning technology, combines historical fault data and the change patterns of new data, and optimizes the learning strategy of the neural network. For example, after several warnings, the system finds that the prediction results of winding insulation aging are highly consistent with the subsequent actual maintenance results, so it adjusts the weight parameters accordingly to improve the credibility of the fault prediction for this category. If a false alarm occurs in a certain prediction, the system will correct the misclassification loss weight of the model to reduce the misjudgment probability of similar situations.

[0061] This application example shows that this system can not only predict equipment failures in advance, but also improve the reliability of prediction through an intelligent optimization mechanism, and combine visualization warning and decision support functions to help the power operation and maintenance team optimize the maintenance strategy, reduce operating costs, and improve the safety and stability of the power system.

[0062] In summary, the present invention relates to a power equipment fault prediction method and related equipment based on multi-modal data. The method includes data collection, data preprocessing, feature extraction, fault prediction modeling, intelligent optimization, and warning mechanism. A variety of sensors are used to collect electrical signals, mechanical vibrations, thermal imaging, sound, and environmental data. Deep learning models are used to extract key features, and convolutional neural networks, long short-term memory networks, and Transformers are combined for multi-modal data fusion. A fault prediction model is constructed based on a hybrid neural network. The model is optimized through Bayesian optimization and reinforcement learning to improve the prediction accuracy and generalization ability. The system provides real-time fault warnings and maintenance suggestions, improves the operation reliability of power equipment, reduces the risk of unplanned outages, and is applicable to the health management of substations, high-voltage transmission equipment, and wind turbine generators.

[0063] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above-mentioned system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0064] In the above embodiments, the descriptions of the various embodiments each have their own emphasis. For parts not described in detail or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0065] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present invention can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0066] In the embodiments provided by the present invention, it should be understood that the disclosed device / terminal and method can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical, mechanical or other forms.

[0067] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0068] In addition, the functional units in the various embodiments of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0069] When the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random-access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0070] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0071] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to generate a computer-implemented process, thereby providing instructions for implementing the steps of the process Figure 1 one process or a plurality of processes and / or blocks Figure 1 steps for the functions specified in one block or a plurality of blocks.

[0073] The above is only to illustrate the technical idea of the present invention and should not be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution according to the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for predicting power equipment faults based on multimodal data, characterized in that: The following steps are involved: Collect and pre-process multi-modal operation data of power equipment; Extracting features of the preprocessed multimodal operation data and performing global feature fusion to obtain multimodal features; Inputting the multimodal features into a pre-trained fault prediction model to obtain the fault category and occurrence probability of the power equipment; The fault prediction model is obtained by inputting the multimodal features into a hybrid neural network for training.

2. The method for predicting power equipment faults based on multimodal data according to claim 1, characterized in that: The multimodal operation data includes: electrical signals, mechanical vibrations, thermal imaging, sound data and environmental data; The preprocessing includes: signal noise reduction, time series alignment and data format standardization; The signal noise reduction adopts short-time Fourier transform or wavelet transform to remove high-frequency noise in the vibration signal; the time series alignment adopts interpolation method to keep data with different sampling frequencies synchronized on the time axis.

3. The method for predicting power equipment failure based on multimodal data according to claim 1, characterized in that: Use Transformer to perform global feature fusion on the pre-processed multi-modal operation data features; The characteristics of the pre-processed multi-modal operation data include: local characteristics of vibration signals and thermal imaging data, and time series characteristics of electrical signals and environmental data; The local features of the vibration signal and thermal imaging data are extracted by using a convolutional neural network; the time series features of the electrical signal and environmental data are obtained by using a long short-term memory network modeling; The Transformer adopts a self-attention mechanism to establish global dependencies between multimodal operation data.

4. The method for predicting power equipment faults based on multimodal data according to claim 3, characterized in that: The fault prediction model realizes deep correlation analysis of multimodal features by combining convolutional neural networks, long short-term memory networks and Transformer, and adopts optimization strategies to train the fault prediction model; The optimization strategies include loss function optimization, Bayesian optimization and reinforcement learning optimization; A data enhancement strategy is adopted in the training process; The data enhancement strategy includes time series data expansion and image data enhancement.

5. A power equipment fault prediction system based on multimodal data, characterized in that: include: Data acquisition and preprocessing module, used to collect and preprocess multi-modal operation data of power equipment; The feature extraction and fusion module is used to extract the features of the preprocessed multimodal operation data and perform global feature fusion to obtain multimodal features; The fault prediction module is used to input the multimodal features into a pre-trained fault prediction model based on a hybrid neural network to obtain the fault category and occurrence probability of the power equipment.

6. The power equipment fault prediction system based on multimodal data according to claim 5, characterized in that: The data acquisition and preprocessing module includes a plurality of sensors, and the sensors transmit multi-modal operation data of the power equipment in a wireless or wired manner to realize remote data monitoring; The fault prediction module includes: a convolutional neural network, a long short-term memory network and a Transformer; the convolutional neural network is responsible for extracting local features of vibration signals and thermal imaging data; the long short-term memory network is responsible for time series modeling of electrical signals and environmental data; and the Transformer is responsible for global feature fusion.

7. The power equipment fault prediction system based on multimodal data according to claim 5, characterized in that: It also includes intelligent optimization module and fault warning and visualization module; The intelligent optimization module adopts a reinforcement learning mechanism to dynamically adjust the learning strategy of the fault prediction model according to the prediction error, thereby improving the prediction accuracy of the fault prediction model; The fault warning and visualization module displays the health status, prediction results and maintenance suggestions of the equipment based on the power operation and maintenance platform, and supports the visualization analysis of historical fault data.

8. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform the method of any one of claims 1-4.

9. A computing device, characterized in that include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for executing the method according to any one of claims 1 to 4.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable in the processor, wherein the processor implements the steps of the method for predicting faults of electric power equipment based on multimodal data as described in any one of claims 1 to 4 when executing the computer program.

Images