A multi-modal fusion fault self-diagnosis method for a wind-solar-storage multi-energy complementary system

Abstract

The application provides a multi-modal fusion fault self-diagnosis method for a wind-solar-storage multi-energy complementary system, comprising: collecting operation data of wind power equipment, photovoltaic components and energy storage equipment as observation data for a fault system, and performing alignment processing on each observation data in the time domain, the space domain and the physical domain after unifying the time standard and the space structure; using a fixed-length sliding window to sample the aligned multi-modal observation data to obtain multi-modal fusion data; extracting local features of each modal data and interaction features between each modal, and then fusing them into an operation state expression vector of the system; constructing a fault propagation chain based on a system topology model, finding an abnormal node through a root cause inference method, and identifying a fault equipment by establishing a mapping relationship between the operation state expression vector of the system and the fault propagation chain. The method significantly improves the perception level of early weak faults, and expands the fault diagnosis range from a single point anomaly to the system structure level.

Description

Technical Field

[0001] This application belongs to the field of fault diagnosis technology for wind-solar-storage systems, and particularly relates to a multimodal fusion fault self-diagnosis method for wind-solar-storage multi-energy complementary systems. Background Technology

[0002] With the continuous expansion of wind and solar power capacity, fault monitoring and operational diagnosis have become core aspects affecting the safety of new energy power plants. Typical faults in wind turbines mainly include abnormal gearbox vibration, main bearing wear, decreased generator winding insulation, and blade structural damage. These faults alter the electrical output characteristics of the generator, further affecting the control behavior of the grid-connected inverter. Common faults in photovoltaic arrays, such as hot spots, microcracks, module mismatch, and combiner box anomalies, can cause a rapid increase in local temperature, which is then transmitted to the inverter via DC-side fluctuations, amplifying power disturbances from the module level to the entire photovoltaic system. Faults in energy storage batteries, such as thermal runaway, internal short circuits, and excessive aging, can lead to accelerated cell temperature rise and abnormal terminal voltage, while also compromising the dynamic response capability of energy storage and affecting the power smoothing effect of wind and solar power. If these faults are not identified in time, they can easily create a chain reaction in the coupled system, causing local faults to gradually evolve into system-level anomalies.

[0003] In existing renewable energy power plants, fault identification typically relies on data collected by the equipment itself, such as vibration signals, current and voltage waveforms, and operating temperatures on the wind power side; infrared images, current and voltage curves, and module temperatures on the photovoltaic side; and individual cell voltages, temperatures, and gas sensor signals on the energy storage side. Some power plants have introduced methods such as manual inspections, drone image inspections, or control system log analysis to attempt to summarize fault patterns from multi-source information. However, due to the significant differences in time scale, data format, and physical meaning among the multimodal characteristics of wind, solar, and energy storage systems, traditional methods often treat each system as an independent object, constructing diagnostic models separately, lacking deep integration of cross-system information. As a result, fault detection remains at the level of a single device, failing to reveal the coupling relationships between wind, solar, and energy storage, and unable to identify complex fault chains from a system-level perspective.

[0004] In summary, existing technologies generally suffer from problems such as difficulty in uniformly representing multimodal information, lack of system coupling characteristics, difficulty in identifying early weak anomalies, and lack of model adaptability. These shortcomings make it difficult to meet the requirements of wind, solar, and energy storage systems for accurate diagnosis, early warning, and system-level control in the context of high-proportion renewable energy. These deficiencies have become key bottlenecks restricting the safe and stable operation of renewable energy power plants. Summary of the Invention

[0005] In view of this, this application aims to address the problems of multimodal fragmentation, insufficient correlation of multi-source information, invisible weak faults, unutilized system coupling relationships, and lack of model adaptability in wind-solar-storage multi-energy complementary systems. It provides a multimodal fusion fault self-diagnosis method for wind-solar-storage multi-energy complementary systems, which can effectively avoid the problems of lack of interpretability or insufficient adaptability of models in traditional methods, and provide sustainable technical support for the safe operation of complex energy systems.

[0006] This application provides a multimodal fusion fault self-diagnosis method for wind-solar-storage multi-energy complementary systems, including: For wind-solar-storage systems with malfunctions, operational data from wind power equipment, photovoltaic modules, and energy storage equipment are collected as observation data. After unifying the time standard and spatial structure of each observation data, the data are aligned in the time domain, spatial domain, and physical domain. By sampling aligned multimodal observation data using a fixed-length sliding window, multimodal fusion data that can characterize the system's operating state within that time interval is obtained. Extract local features of each modality in the multimodal fusion data, as well as the interaction features between each modality, and then fuse them into a system operation state expression vector; A fault propagation chain is constructed based on the system topology model, and abnormal nodes are found through root cause inference. By establishing a mapping relationship between the system's operating state expression vector and the fault propagation chain, faulty equipment is identified.

[0007] Furthermore, the operating data of the wind power equipment includes: continuous vibration signal, wind turbine speed, generator current and voltage, and nacelle temperature; the operating data of the photovoltaic module includes: DC current and voltage of the photovoltaic array, module temperature distribution, and photovoltaic module thermal imaging image; the operating data of the energy storage device includes: single cell current and voltage, temperature distribution, and state of charge.

[0008] Furthermore, the time standard and spatial structure of the various observation data were unified in the following ways: By introducing a time alignment operator to resample each observation data, the observation data of different modalities can have semantic correspondence at the same time. Then, the image data is converted into vectorized spatial structure features through an image coding mapping function, thereby obtaining a unified time standard and spatial structure for each observation data.

[0009] Furthermore, the observation data are aligned in the time domain, spatial domain, and physical domain using the following methods: A dynamic time warping method is used to align the time series of observation data with a unified time standard under different modes. Spatial alignment is achieved by mapping the spatial structural features of image data to a unified geometric coordinate system through a spatial mapping transformation function. The time-aligned and spatially aligned observation data are mapped to a unified physical feature space.

[0010] Furthermore, the local features of each modal data include: vibration amplitude variation, high-frequency energy distribution, and power fluctuation characteristics on the wind power side; voltage and current disturbances, temperature accumulation areas, and hot spot characteristics on the photovoltaic side; individual cell voltage change gradient, temperature rise rate, and load response characteristics on the energy storage side; and spatial structural anomaly characteristics of the image modality.

[0011] Furthermore, the system topology model refers to a system topology model constructed using wind power equipment, photovoltaic modules, energy storage equipment, and key electrical equipment as nodes, and the connection relationships between each device as edges.

[0012] Furthermore, the construction of the fault propagation chain based on the system topology model and the identification of abnormal nodes through root cause inference include: For any node, the propagation strength of the node to other nodes is determined based on the running state expression vectors of the node and other nodes, so as to obtain the propagation weight matrix of the node; then, the faulty node is taken as the propagation starting point, and the fault is propagated step by step in the system topology model according to the propagation weight matrix to generate a fault propagation chain.

[0013] The multimodal fusion fault self-diagnosis method for wind-solar-storage multi-energy complementary systems proposed in this application has the following beneficial effects: By establishing a unified data representation system, multimodal data becomes fusionable; and by combining the local features of observation data with the interaction mechanisms between modes, it can simultaneously perceive short-term impacts and long-term degradation trends, thereby significantly improving the perception level of early-stage minor faults. By introducing a system topology model to model the coupling relationships between devices, the fault diagnosis scope is expanded from single-point anomalies to the system structure level; and by constructing a fault propagation chain to achieve root cause localization, seemingly unrelated anomalies in wind-solar-storage systems are explained within a unified evolutionary framework. This effectively avoids the problems of insufficient interpretability or adaptability of models in traditional methods, providing sustainable technical support for the safe operation of complex energy systems. Attached Figure Description

[0014] Figure 1 A flowchart of a multimodal fusion fault self-diagnosis method for wind-solar-storage multi-energy complementary systems provided in this application embodiment is shown. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this technical solution clearer, the following detailed description, in conjunction with specific embodiments, further illustrates this technical solution. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of this technical solution.

[0016] Please see as follows Figure 1 The flowchart shown is for a multimodal fusion fault self-diagnosis method for wind-solar-storage multi-energy complementary systems. Figure 1 As shown, the method includes: S101. For wind-solar-storage systems with malfunctions, collect operational data from wind power equipment, photovoltaic modules, and energy storage equipment as observation data. After unifying the time standard and spatial structure of each observation data, align the observation data in the time domain, spatial domain, and physical domain.

[0017] The operating data of the wind power equipment includes: continuous vibration signal, wind turbine speed, generator current and voltage, and nacelle temperature; the operating data of the photovoltaic module includes: DC current and voltage of the photovoltaic array, module temperature distribution, and photovoltaic module thermal imaging image; the operating data of the energy storage device includes: individual cell current and voltage, temperature distribution, and state of charge.

[0018] In this step, the wind-solar-storage system includes wind power equipment, photovoltaic modules, and energy storage equipment. The observation data obtained from these devices differ significantly in structural form, physical meaning, and variation patterns. If the observation data are directly spliced ​​together, the spliced ​​result will lack physical meaning. Therefore, this application maps heterogeneous data to a comparable and fusion-compatible unified expression space by synchronously reconstructing and structurally transforming all the original observation data.

[0019] In practice, the time standard and spatial structure of all observation data are unified in the following ways: By introducing a time alignment operator to resample each observation data, the observation data of different modalities can have semantic correspondence at the same time. Then, the image data is converted into vectorized spatial structure features through an image coding mapping function, thereby obtaining a unified time standard and spatial structure for each observation data.

[0020] Here, since wind turbine vibration signals are typically sampled at millisecond-level frequencies, photovoltaic electrical quantities are mostly sampled at second-level frequencies, and energy storage data sampling frequencies fall between the two, a significant time-series misalignment exists between different data sources. To eliminate the impact of time axis inconsistency, this application introduces a time alignment operator to uniformly resample the observation data. This time alignment operator maps observation data from different frequencies of the wind, solar, and energy storage systems to the same time axis, enabling different modal information to have semantic correspondences at the same time. Furthermore, since image data cannot be directly combined with time-series signals, an image encoding mapping function is used to convert image data into vectorized spatial structural features.

[0021] In practice, the following methods are used to align the observation data in the time domain, spatial domain, and physical domain: A dynamic time warping method is used to align the time series of observation data with a unified time standard under different modalities; the spatial structure features of image data are mapped to a unified geometric coordinate system through a spatial mapping transformation function to achieve spatial alignment; and the observation data after time and spatial alignment are mapped to a unified physical feature space.

[0022] In order to eliminate the differences in time scale, spatial structure and physical meaning of multimodal observation data and avoid structural fragmentation between features, thereby improving the effectiveness of subsequent fusion analysis, this application constructs a three-domain alignment mechanism of time domain, spatial domain and physical domain to uniformly align the multimodal features of wind power equipment, photovoltaic modules and energy storage equipment at multiple levels, so that information from different sources has a consistent expression basis.

[0023] As an example, time series alignment is performed using the following time warping method: ; (1) In the formula, These represent the time points of observation data for different modalities. This is the optimal alignment path between observation data of different modalities.

[0024] Time domain alignment enables different data sources to be projected to a unified time index position when the same physical event occurs, effectively alleviating the misalignment problem caused by inconsistent sampling frequencies.

[0025] Furthermore, photovoltaic module thermal imaging images and wind turbine structural images suffer from differences in shooting angle, scale, and equipment structure. To ensure comparability between different images during subsequent feature fusion, this application maps the original image features to a unified geometric coordinate system using a spatial mapping transformation function. Specifically, this is represented as follows: ; (2) In the formula, These are spatially regularized image features. Spatial structural features of the original image; This is a spatial mapping transformation function.

[0026] Spatial domain alignment ensures consistency in pixel distribution and structural location of various image features, enabling comparative analysis of information from different devices or perspectives within the same spatial framework.

[0027] Furthermore, wind power equipment, photovoltaic modules, and energy storage devices belong to different physical systems, and their collected data reflect mechanical, electrical, and electrochemical processes, respectively. Simply aligning numerical values ​​without unifying physical meaning will result in a lack of interpretability in the fusion results. Therefore, this application uses a multimodal physical consistency mapping model to project various features onto a unified physical feature space: ; (3) In the formula, To unify the representation of features in physical space, The feature vector of wind power equipment observation data; For photovoltaic module feature vectors, This represents the feature vector of the energy storage device. This is a physical consistency mapping function.

[0028] By aligning physical operations, fault behaviors of different modes can be analyzed under a unified dimension, providing a theoretical basis for system-level fault inference.

[0029] S102. Using a fixed-length sliding window to sample aligned multimodal observation data, multimodal fusion data that can characterize the system's operating state within the time interval is obtained.

[0030] In this step, after alignment is completed, a fixed-length sliding time window is used to slice the aligned multimodal observation data according to a fixed step size for time series reconstruction.

[0031] As an example, let the length of the sliding window be... Extracting continuous time steps on the time axis with a fixed step size The data segment consists of sampling points, and each time window corresponds to a fused sample containing wind turbine data, photovoltaic data, energy storage data, and image features, which is used to express the operating status of the system within that time interval.

[0032] S103. Extract the local features of each modality data in the multimodal fusion data, as well as the cross-modal interaction features, to fuse them into a system operation state expression vector.

[0033] The local features of each modal data include: vibration amplitude variation, high-frequency energy distribution, and power fluctuation characteristics on the wind power side; voltage and current disturbances, temperature accumulation areas, and hot spot characteristics on the photovoltaic side; single-cell voltage change gradient, temperature rise rate, and load response characteristics on the energy storage side; and spatial structural anomaly characteristics of the image modality.

[0034] As an example, let the input feature vector of the m-th mode at time t be... Its multi-scale local enhancement features are represented as follows: ; (4) In the formula, This represents the local features of the m-th mode. These represent wind power, photovoltaic, and energy storage modes, respectively. For the first Local feature extraction operators at various scales, where K is the number of scales.

[0035] Furthermore, after obtaining the local features of each modal data, a cross-modal attention mechanism is introduced to characterize the coupling relationship between wind, solar, and energy storage, highlighting the correlation factors that significantly impact system operation from different modes. For modes The attention weights are given by the following formula: ; (5) In the formula, Let be the correlation weight matrix of mode m to mode n. Let m be the query vector for modality m; Let n be the key vector of mode n.

[0036] Cross-modal interaction features are represented as follows: ; (6) In the formula, The interaction characteristics between mode m and mode n, Let n be the value vector of mode n.

[0037] Furthermore, the local features of each modality's data are combined with the cross-modal interaction features, and then a fusion function is used to obtain the system's operational state representation vector: ; (7) In the formula, This represents the system-level fault feature vector. It is a multimodal fusion operator.

[0038] During system operation, output is performed within each time window. And the corresponding fault category probability, used to characterize the degree to which the system deviates from the normal state.

[0039] S104. Construct a fault propagation chain based on the system topology model, find abnormal nodes through root cause inference, and identify faulty equipment by establishing a mapping relationship between the system's operating state expression vector and the fault propagation chain.

[0040] The system topology model refers to a system topology model constructed using wind power equipment, photovoltaic modules, energy storage equipment, and key electrical equipment as nodes, and the connection relationships between the equipment as edges.

[0041] As an example, the system topology model is constructed by treating wind power equipment, photovoltaic modules, energy storage equipment, and key electrical equipment as nodes: ; (8) In the formula, Represents a node in the system topology model, i .

[0042] The connection relationships between devices in the system topology model are represented as follows: ; (9) In the formula, This indicates the operational relationships between nodes.

[0043] In practical implementation, the fault propagation chain is constructed based on the system topology model in the following way, and the abnormal nodes are found through root cause inference: For any node, the propagation strength of the node to other nodes is determined based on the running state expression vectors of the node and other nodes, so as to obtain the propagation weight matrix of the node; then, the faulty node is taken as the propagation starting point, and the fault is propagated step by step in the system topology model according to the propagation weight matrix to generate a fault propagation chain.

[0044] As an example, the node propagation strength is calculated by combining the system's running state representation vector: ; (10) in, This represents the propagation strength from node i to node j. Let i be the feature vector of node i; Let j be the feature vector of node j. This is the propagation intensity mapping function.

[0045] Then, by propagating the fault step by step in the system topology model, the fault propagation chain is obtained, which is the set of fault nodes: ; (11) The above methods can be used to locate root causes and infer the scope of their influence.

[0046] As an example, faulty devices in the system under test can be identified in the following way: Set at time The system's operating state representation vector is: ;(12) The characteristics of a fault propagation chain are: (13) Based on the operational status representation vector and the fault propagation chain, the faulty node is deduced from the propagation weight matrix of the node in the system topology model, so as to identify the faulty equipment.

[0047] The above content is only a preferred embodiment of the present invention. For those skilled in the art, many changes can be made in the specific implementation and application scope based on the ideas of the present invention. As long as these changes do not depart from the concept of the present invention, they all fall within the protection scope of the present invention.

Claims

1. A multimodal fusion fault self-diagnosis method for wind-solar-storage multi-energy complementary systems, characterized in that, The method includes: For wind-solar-storage systems with malfunctions, operational data from wind power equipment, photovoltaic modules, and energy storage equipment are collected as observation data. After unifying the time standard and spatial structure of each observation data, the data are aligned in the time domain, spatial domain, and physical domain. By sampling aligned multimodal observation data using a fixed-length sliding window, multimodal fusion data that can characterize the system's operating state within that time interval is obtained. Extract local features of each modality in the multimodal fusion data, as well as the interaction features between each modality, and then fuse them into a system operation state expression vector; A fault propagation chain is constructed based on the system topology model, and abnormal nodes are found through root cause inference. By establishing a mapping relationship between the system's operating state expression vector and the fault propagation chain, faulty equipment is identified.

2. The method as described in claim 1, characterized in that, The operating data of the wind power equipment includes: continuous vibration signal, wind turbine speed, generator current and voltage, and nacelle temperature; the operating data of the photovoltaic module includes: DC current and voltage of the photovoltaic array, module temperature distribution, and photovoltaic module thermal imaging image; the operating data of the energy storage device includes: individual cell current and voltage, temperature distribution, and state of charge.

3. The method as described in claim 1, characterized in that, The time standard and spatial structure of the various observation data are unified in the following ways: By introducing a time alignment operator to resample each observation data, the observation data of different modalities can have semantic correspondence at the same time. Then, the image data is converted into vectorized spatial structure features through an image coding mapping function, thereby obtaining a unified time standard and spatial structure for each observation data.

4. The method as described in claim 1, characterized in that, The observation data are aligned in the time domain, spatial domain, and physical domain using the following methods: A dynamic time warping method is used to align the time series of observation data with a unified time standard under different modes. Spatial alignment is achieved by mapping the spatial structural features of image data to a unified geometric coordinate system through a spatial mapping transformation function. The time-aligned and spatially aligned observation data are mapped to a unified physical feature space.

5. The method as described in claim 1, characterized in that, The local features of each modal data include: vibration amplitude variation, high-frequency energy distribution, and power fluctuation characteristics on the wind power side; voltage and current disturbances, temperature accumulation areas, and hot spot characteristics on the photovoltaic side; individual cell voltage change gradient, temperature rise rate, and load response characteristics on the energy storage side; and spatial structural anomaly characteristics of the image modality.

6. The method as described in claim 1, characterized in that, The system topology model refers to a system topology model constructed using wind power equipment, photovoltaic modules, energy storage equipment, and key electrical equipment as nodes, and the connection relationships between each device as edges.

7. The method as described in claim 1, characterized in that, The process of constructing a fault propagation chain based on a system topology model and identifying anomalous nodes through root cause inference includes: For any node, the propagation strength of the node to other nodes is determined based on the running state expression vectors of the node and other nodes, so as to obtain the propagation weight matrix of the node; then, the faulty node is taken as the propagation starting point, and the fault is propagated step by step in the system topology model according to the propagation weight matrix to generate a fault propagation chain.

Images