Multi-modal data fusion and intelligent analysis method and system for advanced early warning of wind and light storage equipment

Abstract

The invention discloses a multi-modal data fusion and intelligent analysis method and system for advanced early warning of wind and light storage equipment, and the method comprises the steps: deploying data collection equipment, synchronizing the data collection equipment to a virtual equipment model through a digital twin interface, and generating a physical enhancement feature; recombining the physical enhancement features into a three-dimensional feature tensor; orthogonal constraint tensor decomposition is carried out on the three-dimensional tensor, and cross-scale fault features are extracted; constructing a fault propagation graph network FPGN based on the core tensor, and outputting an early warning decision through graph convolution and Transform coding; according to the early warning deviation, a gradient descent method is adopted to adjust the physical constraint weight, and the model is retrained; the three-dimensional feature tensor recombination realizes unified characterization of multi-modal data, orthogonal constraint tensor decomposition forces different dimension features to be independent, a cross-scale coupling mode of a fault is extracted, and the effects of dimension reduction and efficiency improvement are achieved; according to the FPGA network, graph convolution and Transform time sequence coding are fused, and the propagation intensity of modeling faults in an equipment topology network is realized, so that early warning from local anomalies to system-level risks is realized.

Description

Technical Field

[0001] This invention belongs to the field of safety early warning technology, specifically relating to a multimodal data fusion and intelligent analysis method and system for early warning of wind, solar and energy storage equipment. Background Technology

[0002] With the increasing prevalence of integrated wind, solar, and energy storage power stations, downtime losses due to equipment failures have risen sharply. Traditional monitoring methods suffer from three major bottlenecks: Multimodal data fragmentation: Heterogeneous data such as vibration, infrared, and electrochemical data (with different sampling frequencies, dimensions, and physical meanings) are difficult to integrate effectively. Existing methods often use simple feature stitching and ignore spatiotemporal correlation.

[0003] Cross-scale fault propagation lag: Local faults (such as gearbox cracks) can propagate between multiple devices through mechanical / electrical coupling, and traditional single-point detection cannot capture the dynamic propagation path.

[0004] While current digital twin technology can construct virtual models, it has not yet solved core problems such as spatiotemporal alignment of multi-source data, embedding of physical constraints, and extraction of cross-scale correlation features. There is an urgent need for a multimodal early warning method that integrates physical mechanisms and intelligent analysis. Summary of the Invention

[0005] The purpose of this invention is to provide a multimodal data fusion and intelligent analysis method and system for early warning of wind, solar and energy storage equipment, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a multimodal data fusion and intelligent analysis method for early warning of wind, solar and energy storage equipment, comprising the following steps: S1: Deploy data acquisition equipment to collect data from wind turbines, photovoltaics, and energy storage devices, and synchronize it to the virtual device model through the digital twin interface to generate physical augmentation features; S2: Load device physical constraints; S3: Reorganize the physical enhancement features into a three-dimensional feature tensor according to the time scale, spatial scale, and physical dimension; S4: Perform orthogonal constrained tensor decomposition on the three-dimensional tensor to obtain the core tensor and the orthogonal matrix of time, space and physical quantities, and extract cross-scale fault features; S5: Construct a fault propagation graph network FPGN based on core tensors, and output early warning decisions through graph convolution and Transformer encoding; S6: Adjust the physical constraint weights using gradient descent based on the warning deviation, and retrain the model.

[0007] Preferably, in S1, the digital twin interface receives sensor data streams via the OPC UA protocol and uses the IEEE 1588 protocol to achieve multi-source data time synchronization; the virtual device model is generated by driving the device's three-dimensional geometric model, physical equation set, and real-time sensor data.

[0008] Preferably, the physical enhancement feature generation in S1 includes: fusing measured vibration waveforms and fluid dynamics simulation data to correct the aerodynamic load model of the wind turbine blades; and superimposing infrared thermal images and photovoltaic electroluminescent images to mark hot spot areas.

[0009] Preferably, the construction of the three-dimensional feature tensor in S3 includes: Time scale: Interpolate data from different acquisition frequencies to a unified time axis; Spatial scale: Positioning the device in a three-dimensional coordinate system through coordinate mapping; Physical dimension: Normalize the physical quantities of temperature and vibration to the [0,1] interval.

[0010] Preferably, the orthogonal constraint tensor decomposition in S4 satisfies: 3D tensor = core tensor × time matrix × space matrix × physical quantity matrix; The time matrix, space matrix, and physical quantity matrix are orthogonal to each other and are optimized alternately.

[0011] Preferably, the FPGN construction in S5 includes; 5.1 Calculate the fault propagation strength between nodes: ; 5.2 Establish topology connections based on the equipment CAD model; 5.3 A two-layer GATv2 graph convolutional layer is used, with 8 attention heads in each layer and an output dimension of 64. Parallel first-order and second-order convolutional kernels are used to capture the propagation path.

[0012] Preferably, in S5, the early warning decision is made by applying a Transformer encoder to the node feature sequence and fusing spatiotemporal evolution features to output the fault probability.

[0013] A multimodal data fusion and intelligent analysis system for early warning of wind, solar, and energy storage equipment includes: Data Augmentation Module: Used to deploy data acquisition equipment to collect data from wind turbines, photovoltaic and energy storage devices, synchronize it to the virtual device model through the digital twin interface, and generate physical augmentation features; Constraint module: Used to load physical constraints of the device; Feature Tensor Reorganization Module: Used to reorganize physically enhanced features into three-dimensional feature tensors according to time scale, spatial scale, and physical dimension; Tensor decomposition module: used to perform orthogonal constrained tensor decomposition on three-dimensional tensors to obtain core tensors and orthogonal matrices of time, space and physical quantities, and extract cross-scale fault features; Fault Propagation Graph Network Construction Module: Constructs a Fault Propagation Graph Network (FPGN) based on core tensors, and outputs early warning decisions through graph convolution and Transformer encoding; Optimization module: Adjust the physical constraint weights using gradient descent based on the warning deviation, and retrain the model.

[0014] The technical effects and advantages of this invention are as follows: 1. Three-dimensional feature tensor reconstruction realizes unified representation of multimodal data, orthogonal constraint tensor decomposition forces features of different dimensions to be independent, extracts cross-scale coupling patterns of faults, and achieves the effect of dimensionality reduction and efficiency improvement. By embedding physical constraints such as wind shear and vibration coupling of wind turbines, photovoltaic mismatch loss, and battery entropy change model, the feature search space is limited, making the model conform to the equipment operation mechanism and reducing the false alarm rate. By fusing graph convolution and Transformer temporal coding in FPGN networks and modeling the propagation intensity of faults in the device topology network, early warning from local anomalies to system-level risks can be achieved. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0016] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted that these descriptions are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0017] This invention provides, for example Figure 1 The method for early warning of wind, solar and energy storage equipment using multimodal data fusion and intelligent analysis, as shown, includes the following steps: Step S1: Deploy data acquisition equipment, including wind turbine vibration sensors, photovoltaic infrared thermal imagers, and energy storage battery voltage / temperature probes, to collect vibration waveforms, infrared images, and electrochemical data, and synchronize them to the virtual device model through a digital twin interface to generate physical enhancement features; Specifically, the raw data streams (vibration waveforms, infrared images, electrochemical data, and meteorological data) from the sensor wind turbine vibration sensor, photovoltaic infrared thermal imager, energy storage battery voltage / temperature probe, and environmental monitoring station are pushed to the digital twin engine via the OPC UA protocol. The digital twin engine adopts the IEEE 1588 Precise Time Protocol (PTP) to ensure time consistency of multi-source data. Specifically, a virtual device model is a digital twin driven by both physical mechanisms and real-time data. It consists of a three-dimensional geometric model of the device, a set of physical equations for device operation, and real-time sensor data. For example, the virtual device model of a wind turbine consists of a high-precision CAD mesh of the gearbox, blade momentum equations and crack propagation equations, and real-time data detected by wind turbine vibration sensors. Specifically, the generated physical enhancement features are achieved by fusing measured vibration waveforms with fluid dynamics simulation data to correct the aerodynamic load model of the wind turbine blades; and by overlaying infrared thermal images with photovoltaic electroluminescent images to mark hot spot areas.

[0018] Step S2: Apply physical constraints to wind turbines, photovoltaic systems, and energy storage devices; The physical constraints of the wind turbine are based on a CFD-simulated coupled model of wind shear and tower vibration, which constrains the correlation between the frequency domain characteristics of vibration and wind speed. Physical constraints of photovoltaics, calculation of photovoltaic array mismatch loss based on PVMismatch model, and constraint of the mapping relationship between infrared thermal image temperature gradient and output power; The physical constraints of energy storage, based on electrochemical-thermal coupling models (such as the Doyle-Fuller-Newman model), constrain the entropy change characteristics of the voltage / temperature curve; Step S3: Reorganize the physical enhancement features of S1 according to the three-dimensional scale to construct a three-dimensional feature tensor, including the time scale, spatial scale and physical dimension. Specifically, the time scale unifies the data of different acquisition frequencies to the time axis. The establishment of the spatial three-dimensional coordinate system is to position each device in the three-dimensional coordinate system using the coordinate mapping method. The physical dimension is to display the physical quantity features of each device in a normalized way, such as normalizing data such as temperature and fan vibration to the [0, 1] interval. Step S4: Perform orthogonal constrained tensor decomposition on the 3D reconstructed data to extract cross-scale correlated fault features; the specific steps are as follows: Input a three-dimensional feature tensor; The three-dimensional tensor is decomposed into several core parts, including a core tensor and three matrices (corresponding to the three dimensions of time, space and physical quantity respectively). The decomposed form is: three-dimensional tensor = core tensor × time matrix × space matrix × physical quantity matrix, where "×" represents the product of tensor and matrix, which is the product in different dimensions. The three matrices obtained from the decomposition (time matrix, spatial matrix, and physical quantity matrix) are all orthogonal matrices. An orthogonal matrix means that the column vectors of the matrix are mutually perpendicular (the inner product is 0) and have a length of 1. The purpose is to ensure that the features in each dimension after decomposition are independent of each other, avoiding interference between information at different scales. Specifically: the time matrix is ​​orthogonal to ensure that features at different time scales are independent; the spatial matrix is ​​orthogonal to ensure that features at different spatial locations (such as gearboxes and bearings) are independent; and the physical quantity matrix is ​​orthogonal to ensure that features of different physical quantities (such as vibration and temperature) are independent. Decomposition process: The three matrices (time matrix, space matrix, and physical quantity matrix) are optimized alternately, that is, two of the matrices are fixed and the third matrix is ​​optimized. This process is repeated until convergence. In each optimization step, the least squares method is used to solve the problem, and singular value decomposition (SVD) is used to force the matrices to be orthogonal. After decomposition, the output is a core tensor and three orthogonal matrices. At this point, the core tensor contains cross-scale correlation features and represents the fault features. Its dimension is much smaller than that of the original tensor, which plays a role in dimensionality reduction. The three orthogonal matrices correspond to the time, space and physical quantity dimensions, respectively, and they represent the feature distribution of the core tensor in each dimension. Using the decomposed core tensor as the basis for subsequent fault diagnosis, each element in the core tensor represents the correlation strength of features in three dimensions: time, space, and physical quantity. We can discover faults by analyzing outliers in the core tensor.

[0019] Step S5: Construct a fault propagation graph network (FPGN) and output early warning decisions based on graph convolution; Specifically, first calculate the fault propagation strength: , in, The values ​​represent the fault propagation strength, W1 and W2 are the weights of the neural network, b1 and b2 represent the biases, σ is the sigmoid activation function, and hi and hj represent the feature vectors of node i and node j, respectively. These feature vectors are extracted from the core tensor obtained in the previous step. Indicates physical distance. This shows how to concatenate two feature vectors and the reciprocal of the distance into a longer vector; Then, a basic connection is established between the calculated fault propagation intensity and the equipment CAD model; It adopts a two-layer GATv2 architecture, with 8 attention heads in each layer and an output dimension of 64. First, a basic convolution is established. Then, the features of adjacent nodes are weighted through an attention mechanism. Next, multi-scale convolution is used to capture local and global propagation patterns by parallel first-order (adjacent nodes) and second-order (adjacent nodes' adjacent nodes) convolution kernels. Finally, features are fused through a gating mechanism to achieve the graph convolution layer design. To simultaneously capture the temporal evolution and spatial propagation of faults, a Transformer encoder is applied to the node feature sequence to achieve fault early warning, and then decisions are made based on the early warning results.

[0020] Step S6: When the warning result deviates from the actual fault, the gradient descent method is used to adjust the object weights and retrain the model.

[0021] This invention also provides a multimodal data fusion and intelligent analysis system for early warning of wind, solar and energy storage equipment, comprising: Data Augmentation Module: Used to deploy data acquisition equipment to collect data from wind turbines, photovoltaic and energy storage devices, synchronize it to the virtual device model through the digital twin interface, and generate physical augmentation features; Constraint module: Used to load physical constraints of the device; Feature Tensor Reorganization Module: Used to reorganize physically enhanced features into three-dimensional feature tensors according to time scale, spatial scale, and physical dimension; Tensor decomposition module: used to perform orthogonal constrained tensor decomposition on three-dimensional tensors to obtain core tensors and orthogonal matrices of time, space and physical quantities, and extract cross-scale fault features; Fault Propagation Graph Network Construction Module: Constructs a Fault Propagation Graph Network (FPGN) based on core tensors, and outputs early warning decisions through graph convolution and Transformer encoding; Optimization module: Adjust the physical constraint weights using gradient descent based on the warning deviation, and retrain the model.

[0022] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A multimodal data fusion and intelligent analysis method for early warning of wind, solar, and energy storage equipment, characterized in that, Includes the following steps: S1: Deploy data acquisition equipment to collect data from wind turbines, photovoltaics, and energy storage devices, and synchronize it to the virtual device model through the digital twin interface to generate physical augmentation features; S2: Load device physical constraints; S3: Reorganize the physical enhancement features into a three-dimensional feature tensor according to the time scale, spatial scale, and physical dimension; S4: Perform orthogonal constrained tensor decomposition on the three-dimensional tensor to obtain the core tensor and the orthogonal matrix of time, space and physical quantities, and extract cross-scale fault features; S5: Construct a fault propagation graph network FPGN based on core tensors, and output early warning decisions through graph convolution and Transformer encoding; S6: Adjust the physical constraint weights using gradient descent based on the warning deviation, and retrain the model.

2. The multimodal data fusion and intelligent analysis method for early warning of wind, solar and energy storage equipment according to claim 1, characterized in that: The digital twin interface in S1 receives sensor data streams via the OPC UA protocol and uses the IEEE 1588 protocol to achieve time synchronization of multi-source data; the virtual device model is generated by driving the device's three-dimensional geometric model, physical equation set, and real-time sensor data.

3. The multimodal data fusion and intelligent analysis method for early warning of wind, solar and energy storage equipment according to claim 1, characterized in that: The physical enhancement feature generation in S1 includes: fusing measured vibration waveforms and fluid dynamics simulation data to correct the aerodynamic load model of the wind turbine blades; and overlaying infrared thermal images and photovoltaic electroluminescence images to mark hot spot areas.

4. The multimodal data fusion and intelligent analysis method for early warning of wind, solar and energy storage equipment according to claim 1, characterized in that: The construction of the three-dimensional feature tensor in S3 includes: Time scale: Interpolate data from different acquisition frequencies to a unified time axis; Spatial scale: Positioning the device in a three-dimensional coordinate system through coordinate mapping; Physical dimension: Normalize the physical quantities of temperature and vibration to the [0,1] interval.

5. The multimodal data fusion and intelligent analysis method for early warning of wind, solar and energy storage equipment according to claim 1, characterized in that: The orthogonal constraint tensor decomposition in S4 satisfies: 3D tensor = core tensor × time matrix × space matrix × physical quantity matrix; The time matrix, space matrix, and physical quantity matrix are orthogonal to each other and are optimized alternately.

6. The multimodal data fusion and intelligent analysis method for early warning of wind, solar and energy storage equipment according to claim 1, characterized in that: The FPGN construction in S5 includes: 5.1 Calculate the fault propagation strength between nodes: ； 5.2 Establish topology connections based on the equipment CAD model; 5.3 A two-layer GATv2 graph convolutional layer is used, with 8 attention heads in each layer and an output dimension of 64. Parallel first-order and second-order convolutional kernels are used to capture the propagation path.

7. The multimodal data fusion and intelligent analysis method for early warning of wind, solar and energy storage equipment according to claim 1, characterized in that: In S5, the early warning decision is achieved by applying a Transformer encoder to the node feature sequence and fusing spatiotemporal evolution features to output the fault probability.

8. A multimodal data fusion and intelligent analysis system for early warning of wind, solar, and energy storage equipment, characterized in that, include: Data Augmentation Module: Used to deploy data acquisition equipment to collect data from wind turbines, photovoltaic and energy storage devices, synchronize it to the virtual device model through the digital twin interface, and generate physical augmentation features; Constraint module: Used to load physical constraints of the device; Feature Tensor Reorganization Module: Used to reorganize physically enhanced features into three-dimensional feature tensors according to time scale, spatial scale, and physical dimension; Tensor decomposition module: used to perform orthogonal constrained tensor decomposition on three-dimensional tensors to obtain core tensors and orthogonal matrices of time, space and physical quantities, and extract cross-scale fault features; Fault Propagation Graph Network Construction Module: Constructs a Fault Propagation Graph Network (FPGN) based on core tensors, and outputs early warning decisions through graph convolution and Transformer encoding; Optimization module: Adjust the physical constraint weights using gradient descent based on the warning deviation, and retrain the model.

Images