A multi-modal data fusion intelligent inspection method for a hydropower plant

Abstract

The application discloses a kind of for hydropower plant multimodal data fusion intelligent inspection method, belong to hydropower plant intelligent inspection technical field.This method obtains vibration signal, temperature signal, ultrasonic signal and oil frequency spectrum signal in the same time window, with vibration signal main frequency component as benchmark reference axis, temperature signal, ultrasonic signal and oil frequency spectrum signal are aligned in time domain phase, generate time delay offset sequence and build causal influence relationship diagram;Local similarity is calculated to two physical field signals connected by each edge in the diagram, and fusion weight is distributed according to signal amplitude ratio, and the fusion state characteristic vector is obtained by superposition;The fusion state characteristic vector of continuous N time windows is extracted to form state transition matrix, the real part sequence of eigenvalue is calculated, and the real part positive value is marked as abnormal early warning source point when existing.The method uses the time delay causal structure of multiple physical fields and the state transition divergence characteristics, realizes early abnormal early warning.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology for hydropower plants, specifically to an intelligent inspection method for multimodal data fusion in hydropower plants. Background Technology

[0002] The operating environment of hydropower plant units is complex, with physical quantities such as vibration, temperature, ultrasonic waves, and oil spectrum reflecting equipment status at different time scales. Existing inspection methods independently acquire and threshold-compare signals from each sensor, or directly stitch together multi-channel features before inputting them into a classification model. The independent acquisition and comparison method treats each physical field as an independent time series, failing to extract the propagation delay and causal relationship between different physical fields. Vibration changes caused by early equipment damage propagate to the temperature and oil feature fields with a time difference, which is not compensated for in traditional methods. This leads to misalignment of multi-source signals on the time axis, resulting in contradictory fluctuation components in the fusion result. Some methods extract multi-channel features at the same time and perform normalization or principal component fusion, but the phase relationship between the physical fields is still ignored, and time delay information cannot be incorporated into the subsequent discrimination process. Directly splicing features uses fixed weights or simple statistical weighting for each channel, without dynamically adjusting the weights based on the local similarity of signal morphology and amplitude ratio. When a channel signal only experiences local morphological distortion but the amplitude has not yet exceeded the limit, fixed weighting can mask the change, reducing the sensitivity of fused state features to early faults. Regarding anomaly detection, existing methods mostly rely on threshold exceeding the current window or feature distance exceeding the limit, only identifying already manifested anomalies. They cannot determine whether signal evolution has a divergent trend based on the state transition relationship across multiple consecutive time windows, limiting their ability to warn of early latent faults. The propagation delay relationship between multi-physics field signals, changes in local morphological similarity, and the state evolution trend of consecutive windows have not been fully utilized, hindering the transformation of hydropower plant inspections from single-point alarms to trend-based early warnings. Summary of the Invention

[0003] To address the issues of time misalignment of multi-physics field signals, fixed fusion weights, and reliance on current amplitude limits for anomaly detection in existing multi-modal inspections of hydropower plants, this paper proposes an intelligent inspection method for multi-modal data fusion in hydropower plants. The method aligns the phase of the multi-physics field signals in the time domain using the dominant frequency component of the vibration signal as the reference axis, extracts the time delay offset, and constructs a causal relationship graph. For each edge connecting two physical field signals in the graph, the local similarity is calculated, and fusion weights are allocated according to the signal amplitude ratio to obtain a fusion state feature vector. A state transition matrix is ​​formed by extracting the fusion state feature vectors from N consecutive time windows. The signal evolution divergence trend is determined based on the sign of the real part of the eigenvalues, and anomaly warning source points are marked.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The present invention discloses an intelligent inspection method for multimodal data fusion in hydropower plants, based on intrinsic mode reconstruction and causal tracing chain construction of physical field signals, including: acquiring the original signal sequences of multiple physical sensors within the same time window, wherein the original signal sequences include vibration signals, temperature signals, ultrasonic signals, and oil spectrum signals; using the dominant frequency component of the vibration signal as a reference axis, aligning the temperature signal, ultrasonic signal, and oil spectrum signal with the vibration signal in the time domain phase to generate a time delay offset sequence between each physical field; constructing a causal influence relationship diagram of the multi-physical field signals on the time axis based on the time delay offset sequence, wherein... In the causal relationship diagram, each node represents a sampling channel of a different physical field, and each edge represents the time delay offset between two channel signals. For the two physical field signals connected by each edge in the causal relationship diagram, the local similarity of the two signals in terms of signal shape is calculated, and the fusion weight is allocated according to the signal amplitude ratio. The fusion weight is multiplied by the corresponding physical field signal and then superimposed by channel to obtain the fusion state feature vector of the monitoring point within the time window. The fusion state feature vectors of the N consecutive time windows before the current time window are extracted to form a state transition matrix. The sequence of real parts of the eigenvalues ​​of the state transition matrix is ​​calculated, and it is determined whether there are positive real parts. If there are positive real parts, the monitoring point is marked as an abnormal warning source point.

[0005] As a technical solution of this invention, when acquiring the raw signal sequences of multiple physical sensors within the same time window, vibration sensors, thermocouple sensors, ultrasonic probe transmitter-receiver pairs, and online oil spectrometers are deployed at key locations of each unit in the hydropower plant. A unified sampling clock triggers each sensor to begin acquisition at the same time, with the acquisition duration equal to an integer multiple of the unit's rotation period. The raw signals output by each sensor are converted from analog to digital and packaged into data frames. Each frame contains a sensor identifier, a timestamp, and a signal value sequence. A unified sampling clock and full-cycle acquisition improve the temporal synchronization of the physical field signals and reduce the impact of truncation errors on subsequent phase alignment and causal analysis.

[0006] Preferably, when using the dominant frequency component of the vibration signal as the reference axis and performing time-domain phase alignment, an autocorrelation operation is performed on the vibration signal to extract the period value corresponding to the peak value of the autocorrelation function, and the peak frequency is defined as the dominant frequency component. Cross-correlation operations are then performed on the temperature signal, ultrasonic signal, and oil spectrum signal with the vibration signal, and the hysteresis time value corresponding to the peak value of each cross-correlation function is extracted as the initial time delay offset of each physical field relative to the vibration signal. After sequentially shifting the vibration signal along the time axis by the initial time delay offset corresponding to each physical field, signal segments within the overlapping time period are re-extracted from the signals of each physical field to complete the time-domain phase alignment. Alignment based on the dominant vibration frequency highlights the vibration transmission path under rotating machinery faults and eliminates phase deviations caused by response delays of different sensors.

[0007] As a technical solution of this invention, when constructing the causal relationship graph, the vibration sensor channel, thermocouple sensor channel, ultrasonic probe channel, and oil spectrometer channel are respectively set as four nodes in the causal relationship graph. Under the condition that there is a measurable cross-correlation peak between two nodes and the peak amplitude is greater than a preset detection threshold, a directed edge is established between the two nodes. The direction of the edge is from the node used as the reference signal in the cross-correlation calculation to the node used as the hysteresis signal, and the weight value of the edge is equal to the corresponding time delay offset. If the cross-correlation peak amplitude between two nodes is not greater than the preset detection threshold, no edge is established between the two nodes. Threshold filtering can suppress spurious connections caused by background noise, making the causal graph more realistically reflect the propagation relationship between physical fields.

[0008] Preferably, when calculating local similarity and assigning fusion weights, signal segments corresponding to the time points of two physical field signals are extracted along the time axis using a sliding window approach. Normalization is performed on the two signal segments within each window. The Euclidean distance between the two normalized signal segments is calculated, and the reciprocal of the Euclidean distance is used as the local similarity for that window. The average of the local similarities across all windows is taken as the overall morphological similarity between the two physical field signals connected by that edge, and this overall morphological similarity is used as the base for assigning fusion weights. This method emphasizes channels with consistent signal morphology and are sensitive to anomalies, reducing the interference of amplitude abrupt changes on the fusion weights.

[0009] Preferably, when generating the fused state feature vector, the fusion weights corresponding to all edges in the causal influence graph are multiplied by the physical field signal amplitude sequences of the target nodes they point to, obtaining the weighted signal components contributed by each edge; all weighted signal components pointing to the same target node are summed with the original signal amplitude sequences of that target node itself to obtain the fused signal amplitude sequence of that target node; the fused signal amplitude sequences of all target nodes are concatenated in node number order to form the fused state feature vector. Through causal weighting and channel superposition, the multi-physics signals are compressed into comparable state features, while preserving the causal contribution relationships between nodes.

[0010] As a technical solution of this invention, when marking anomaly warning source points, the fused state feature vectors of every two adjacent time windows in N consecutive time windows are formed into a transition pair. All transition pairs are arranged in chronological order to construct a state transition matrix, where the number of rows in the state transition matrix is ​​the feature vector dimension and the number of columns is the number of transition pairs. The state transition matrix is ​​then subjected to eigenvalue decomposition to obtain all feature values. The real part of each feature value is extracted to form a real part sequence. Each real part value in the real part sequence is scanned. When at least one real part value is greater than zero, it is determined that the signal evolution process of the monitoring point has a divergence trend, and the monitoring point is marked as an anomaly warning source point. By judging the divergence trend of signal evolution through positive real part values, the trend of the system transitioning from stable to unstable can be captured in the early stages of a fault, improving the timeliness of warnings.

[0011] Preferably, during the extraction of the dominant frequency component of the vibration signal, if the autocorrelation function shows multiple equally spaced peaks, the frequency corresponding to the peak with the largest amplitude is selected as the dominant frequency component to reduce the probability of misjudging the dominant frequency caused by harmonics or noise and improve the stability of the phase alignment reference. The preset detection threshold is dynamically set based on the statistical mean and standard deviation of the background noise level of each physical field signal to adapt to noise changes under different operating conditions. The eigenvalues ​​of the state transition matrix are calculated using the power iteration method or QR decomposition method, ensuring both computational accuracy and computational efficiency requirements for online inspection.

[0012] The technical effects and advantages provided by the present invention in the above technical solution are as follows: Using the dominant frequency component of the vibration signal as the reference axis, cross-correlation operations are performed on the temperature signal, ultrasonic signal, and oil spectrum signal to extract the time delay offset of each physical field relative to the vibration signal. A directed causal relationship graph is then established based on the cross-correlation peak value and a preset detection threshold. This processing method explicitly represents the implicit propagation order between different physical fields as directed edges and time delay weights between nodes, restoring the causal correspondence on the time axis before the multi-source signals enter the fusion stage. Compared to traditional methods that directly splice the signals from each channel at the same timestamp or perform fixed phase compensation, this method uses the dominant vibration frequency as a unified reference, preserving the time delay process information of the fault propagation from the vibration source to the temperature field, ultrasonic field, and oil characteristic field. This reduces the interference of time misalignment caused by different propagation lags of multi-physical field signals on the fused state feature vector. The fused state feature vector can thus characterize the causal transmission structure of the same fault event in the multi-physical fields, rather than simply superimposing the amplitudes of each channel, thereby obtaining a more consistent fusion expression in the early, weak fault stage. For each edge connecting two physical field signals in the causal relationship graph, corresponding segments are extracted along the time axis using a sliding window, normalized, and then the Euclidean distance is calculated. The reciprocal of the distance is used as the local similarity, and the average value of the window is taken as the overall morphological similarity. Fusion weights are then allocated according to the signal amplitude ratio. This weight is dynamically adjusted with local changes in signal morphology. When a channel exhibits local morphological distortion but its amplitude does not exceed the limit, its fusion contribution can be redistributed based on the morphological difference, avoiding the averaging effect of fixed weights on subtle morphological changes and ensuring that the fused state feature vector remains sensitive to local defect morphology. The fused state feature vectors from N consecutive time windows form a state transition matrix. The state transition matrix is ​​eigenvalued, and the sequence of real parts of the eigenvalues ​​is extracted. When a positive real part exists, the signal evolution is determined to have a divergent trend, and an anomaly warning source is marked. This method identifies system instability characteristics from the state transition relationships of multiple windows, independent of whether the current amplitude exceeds the threshold. It can provide early warnings for latent anomalies and extends the warning judgment from single-point exceedances to the level of state evolution trends. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0014] Figure 1 This is a flowchart of a multimodal data fusion intelligent inspection method for hydropower plants; Figure 2 This is a flowchart of an anomaly early warning method for monitoring points based on the real part of the eigenvalues ​​of the state transition matrix. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] See Figure 1 This invention provides an intelligent inspection method for multimodal data fusion in hydropower plants, based on intrinsic mode reconstruction of physical field signals and construction of causal tracing chains, including the following processes.

[0017] The system acquires raw signal sequences from multiple physical sensors within the same time window, including vibration, temperature, ultrasonic, and oil spectrum signals. Using the dominant frequency component of the vibration signal as the reference axis, the temperature, ultrasonic, and oil spectrum signals are time-domain phase aligned with the vibration signal to generate a time delay offset sequence between each physical field. Based on the time delay offset sequence, a causal relationship diagram of the multi-physical field signals on the time axis is constructed. Each node in the causal relationship diagram represents a sampling channel of a different physical field, and each edge represents the time delay offset between two channels. For each edge connecting two physical field signals in the causal relationship diagram, the local similarity in signal morphology is calculated, and fusion weights are assigned according to the signal amplitude ratio. The fusion weights are multiplied by the corresponding physical field signals and then superimposed by channel to obtain the fused state feature vector of the monitoring point within the time window. The fused state feature vectors of the N consecutive time windows preceding the current time window are extracted to form a state transition matrix. The sequence of real parts of the eigenvalues ​​of the state transition matrix is ​​calculated, and it is determined whether there are positive real parts. If there are positive real parts, the monitoring point is marked as an abnormal warning source point.

[0018] In practice, raw signal sequences from multiple physical sensors within the same time window are acquired, including vibration signals, temperature signals, ultrasonic signals, and oil spectrum signals. Vibration sensors, thermocouple sensors, ultrasonic transceiver pairs, and online oil spectrometers are deployed at key locations in each unit of the hydropower plant. Vibration sensors are installed in the unit bearing housings, turbine top cover, and generator stator housing. Thermocouple sensors are installed on the thrust bearing pads, guide bearing pads, and stator windings. Ultrasonic transceiver pairs are installed on the turbine volute wall and tailrace wall. The online oil spectrometer is connected to the unit's oil circulation pipeline. All sensor installation locations are mechanically fixed and electrically shielded.

[0019] All sensors are connected to a unified sampling clock. The sampling clock is generated by a synchronization module, which receives satellite timing signals or plant-level synchronization pulse signals to generate periodic sampling trigger pulses. These sampling trigger pulses are distributed to the vibration sensors, thermocouple sensors, ultrasonic probe transmitter-receiver pairs, and the online oil spectrometer via equal-length cables or optical fibers, ensuring that all sensors begin acquiring data at the same time. The rising edge of the sampling trigger pulse is used as the start time of data acquisition.

[0020] The data acquisition duration is set to an integer multiple of the unit's rotation cycle. The unit's rotation cycle is obtained through a key-phase sensor. The key-phase sensor outputs a pulse for each detected rotation of the unit, and the time interval between adjacent pulses is recorded as the unit's rotation cycle. The acquisition duration is obtained by multiplying the unit's rotation cycle by a positive integer multiple. This positive integer multiple is set according to the time window length requirements of the inspection task; when the inspection task requires single-cycle analysis, the positive integer multiple is set to 1. Within the acquisition duration, all sensors acquire data synchronously and continuously.

[0021] The raw signals output by each sensor are packaged into data frames after analog-to-digital conversion. The vibration sensor, thermocouple sensor, and ultrasonic probe transmit / receive pairs output analog voltage or analog current signals, while the online oil spectrometer outputs the analog electrical signal corresponding to the spectral response signal. The analog-to-digital converter samples and quantizes the analog signals from each channel at equal intervals. The sampling rate of the analog-to-digital converter is determined based on the highest effective frequency component of the vibration signal, and the highest effective frequency component of the vibration signal does not exceed half of the sampling rate of the analog-to-digital converter.

[0022] The length of the signal value sequence in a data frame is determined by the analog-to-digital converter's sampling rate, the unit's rotation period, and positive integer multiples. The expression for the length of the signal value sequence in a data frame is: in, This indicates the length of the signal value sequence in the data frame, expressed in the number of sampling points. This indicates the sampling rate of the analog-to-digital converter, measured in Hertz (Hz). This indicates the unit's rotation period, in seconds; This indicates the integer multiple of the rotation cycles included in the data acquisition duration, with a value range of positive integers. It is set according to the time window length of the inspection task. When single-cycle analysis is required... The value is set to 1. The sampling rate of the analog-to-digital converter is determined based on the highest effective frequency component of the vibration signal, and the highest effective frequency component of the vibration signal shall not exceed half of the sampling rate of the analog-to-digital converter. The unit rotation period is determined by the time interval between adjacent pulses measured by the key phase sensor. Positive integer multiples are given by the inspection system configuration parameters.

[0023] Each frame of data contains a sensor identifier, a timestamp, and a sequence of signal values. The sensor identifier is coded to distinguish between vibration sensor channels, thermocouple sensor channels, ultrasonic probe channels, and oil spectrometer channels. The timestamp records the rising edge of the sampling trigger pulse, and its accuracy is consistent with the sampling clock synchronization accuracy. The sequence of signal values ​​is arranged in the order of sampling.

[0024] In practical implementation, the dominant frequency component of the vibration signal is used as the reference axis. The temperature signal, ultrasonic signal, and oil spectrum signal are time-domain phase aligned with the vibration signal to generate a time delay offset sequence between the various physical fields. Autocorrelation is performed on the vibration signal, and the period value corresponding to the peak value of the autocorrelation function is extracted. The peak frequency is defined as the dominant frequency component.

[0025] In some embodiments, the vibration signal is the raw signal sequence acquired by the vibration sensor. The autocorrelation operation is performed using the sliding inner product of the discrete vibration signal sequence. The autocorrelation function achieves a global maximum peak when the hysteresis is zero, and a local peak when the hysteresis is close to the vibration signal period. The period value corresponding to the peak value of the autocorrelation function is determined by searching for peak positions other than zero hysteresis. The hysteresis corresponding to the peak position is taken as the vibration signal period value. The dominant frequency component is calculated from the reciprocal of the vibration signal period value.

[0026] In practice, if the autocorrelation function exhibits multiple equally spaced peaks, the frequency corresponding to the peak with the largest amplitude is selected as the dominant frequency component. The peaks of the autocorrelation function, excluding those with zero lag, are sorted by amplitude, and the period corresponding to the peak with the largest amplitude is used to calculate the dominant frequency component. When multiple equally spaced peaks appear, the lag difference between adjacent peaks is equal.

[0027] In some embodiments, cross-correlation operations are performed between the temperature signal, the ultrasonic signal, and the oil spectrum signal and the vibration signal, respectively. The cross-correlation operation between the temperature signal and the vibration signal is performed by the inner product of the discrete sequences of the temperature signal and the vibration signal under different hysteresis values. Similarly, the cross-correlation operation between the ultrasonic signal and the vibration signal is performed by the inner product of the discrete sequences of the ultrasonic signal and the vibration signal under different hysteresis values. Finally, the cross-correlation operation between the oil spectrum signal and the vibration signal is performed by the inner product of the discrete sequences of the oil spectrum signal and the vibration signal under different hysteresis values.

[0028] In practical implementation, the lag time values ​​corresponding to the peak values ​​of each cross-correlation function are extracted as the initial time delay offset of each physical field relative to the vibration signal. Specifically, the lag time value corresponding to the peak value of the cross-correlation function between the temperature signal and the vibration signal is used as the initial time delay offset of the temperature signal relative to the vibration signal. Similarly, the lag time value corresponding to the peak value of the cross-correlation function between the ultrasonic signal and the vibration signal is used as the initial time delay offset of the ultrasonic signal relative to the vibration signal. Finally, the lag time value corresponding to the peak value of the cross-correlation function between the oil spectrum signal and the vibration signal is used as the initial time delay offset of the oil spectrum signal relative to the vibration signal.

[0029] In some embodiments, the peak search range of the cross-correlation function is limited to a hysteresis value ranging from zero to half the acquisition time. A condition where the peak amplitude of the cross-correlation function exceeds a preset detection threshold is used to confirm the validity of the peak. The hysteresis value corresponding to the peak of the cross-correlation function is obtained by multiplying the hysteresis value by the sampling interval, where the sampling interval is the reciprocal of the sampling rate of the analog-to-digital converter.

[0030] In practical implementation, the vibration signal is sequentially shifted along the time axis by the initial time delay offset corresponding to each physical field, and then signal segments within the overlapping time period are re-extracted with the signals of each physical field to achieve time-domain phase alignment. The initial time delay offset corresponding to the temperature signal is used to shift the position of the vibration signal along the time axis; the shifted vibration signal and temperature signal then extract signal segments within the overlapping time period. Similarly, the initial time delay offset corresponding to the ultrasonic signal is used to shift the position of the vibration signal along the time axis; the shifted vibration signal and ultrasonic signal then extract signal segments within the overlapping time period. Finally, the initial time delay offset corresponding to the oil spectrum signal is used to shift the position of the vibration signal along the time axis; the shifted vibration signal and oil spectrum signal then extract signal segments within the overlapping time period.

[0031] In some embodiments, the translation direction is determined by the sign of the initial time delay offset. When the initial time delay offset is positive, the translated vibration signal moves in the positive direction of the time axis by the absolute value of the initial time delay offset. When the initial time delay offset is negative, the translated vibration signal moves in the negative direction of the time axis by the absolute value of the initial time delay offset. The start time of the signal segment within the overlapping time period is determined by the later start time of the translated vibration signal and the corresponding physical field signal, and the end time of the signal segment within the overlapping time period is determined by the earlier end time of the translated vibration signal and the corresponding physical field signal.

[0032] In practical implementation, the time delay offset sequence includes the lag time values ​​of the temperature signal relative to the vibration signal, the ultrasonic signal relative to the vibration signal, and the oil spectrum signal relative to the vibration signal. The lag time value of the temperature signal relative to the vibration signal is determined by the lag time value corresponding to the peak value of the cross-correlation function between the temperature and vibration signals. The lag time value of the ultrasonic signal relative to the vibration signal is determined by the lag time value corresponding to the peak value of the cross-correlation function between the ultrasonic and vibration signals. The lag time value of the oil spectrum signal relative to the vibration signal is determined by the lag time value corresponding to the peak value of the cross-correlation function between the oil spectrum and vibration signals. The time delay offset sequence is arranged in the order of temperature signal, ultrasonic signal, and oil spectrum signal.

[0033] In practical implementation, a causal relationship diagram of multi-physics field signals on the time axis is constructed based on the time delay offset sequence. Each node in the causal relationship diagram represents a sampling channel of a different physical field, and each edge represents the time delay offset between two channel signals. The vibration sensor channel, thermocouple sensor channel, ultrasonic probe channel, and oil spectrometer channel are each set as a node in the causal relationship diagram. The vibration sensor channel corresponds to the vibration signal, the thermocouple sensor channel to the temperature signal, the ultrasonic probe channel to the ultrasonic signal, and the oil spectrometer channel to the oil spectrum signal. The node identifier for the vibration sensor channel is set to V, the node identifier for the thermocouple sensor channel is set to T, the node identifier for the ultrasonic probe channel is set to U, and the node identifier for the oil spectrometer channel is set to O. The node identifier is used to uniquely identify each node in the causal relationship diagram.

[0034] In some embodiments, cross-correlation is performed on any two of the four nodes, with the two physical field signal sequences corresponding to the two nodes being the first signal sequence and the second signal sequence, respectively. The first signal sequence is taken as the reference signal, and the second signal sequence is taken as the hysteresis signal. The cross-correlation operation is performed on the hysteresis sequence, which ranges from zero to half the acquisition time, with the hysteresis step size being the sampling interval of the analog-to-digital converter. For each hysteresis, the reference signal sequence is kept stationary, and the hysteresis signal sequence is shifted forward along the time axis by the hysteresis. The inner product of the reference signal sequence and the shifted hysteresis signal sequence within the overlapping interval is calculated, and the inner product value is taken as the value of the cross-correlation function at the hysteresis. The cross-correlation function is searched for values ​​at all hysteresis positions, and the position with the largest amplitude is taken as the cross-correlation peak position, with the cross-correlation peak amplitude used as the judgment criterion.

[0035] Optionally, for the cross-correlation operation between the vibration sensor channel and the thermocouple sensor channel, the first signal sequence is the vibration signal sequence, and the second signal sequence is the temperature signal sequence. For the cross-correlation operation between the vibration sensor channel and the ultrasonic probe channel, the first signal sequence is the vibration signal sequence, and the second signal sequence is the ultrasonic signal sequence. For the cross-correlation operation between the vibration sensor channel and the oil spectrometer channel, the first signal sequence is the vibration signal sequence, and the second signal sequence is the oil spectrum signal sequence. For the cross-correlation operation between the thermocouple sensor channel and the ultrasonic probe channel, the first signal sequence is the temperature signal sequence, and the second signal sequence is the ultrasonic signal sequence. For the cross-correlation operation between the thermocouple sensor channel and the oil spectrometer channel, the first signal sequence is the temperature signal sequence, and the second signal sequence is the oil spectrum signal sequence. For the cross-correlation operation between the ultrasonic probe channel and the oil spectrometer channel, the first signal sequence is the ultrasonic signal sequence, and the second signal sequence is the oil spectrum signal sequence.

[0036] In practice, the preset detection threshold is dynamically set based on the statistical mean and standard deviation of the background noise level of each physical field signal. During idle operation or under historical normal operating conditions, raw signal sequences for each physical field signal are collected over multiple time windows. The arithmetic mean of the signal amplitude within each time window is calculated to obtain the window mean. The standard deviation of the signal amplitude within each time window is calculated to obtain the window standard deviation. The arithmetic mean of the window means across multiple time windows is calculated to obtain the statistical mean of the background noise level. The arithmetic mean of the window standard deviations across multiple time windows is calculated to obtain the statistical standard deviation of the background noise level. The preset detection threshold is determined by the following formula: in, This indicates the preset detection threshold, expressed in units of signal amplitude. The statistical mean of the background noise level, expressed in units of signal amplitude; The statistical standard deviation of the background noise level, expressed in units of signal amplitude; This represents the threshold confidence coefficient, and its value ranges from positive real numbers. Determined based on the confidence level of the background noise amplitude distribution under historical normal operating conditions, when the background noise amplitude approximately follows a Gaussian distribution. The value of 3 is based on the standard deviation multiple corresponding to the 99.7% confidence interval under a Gaussian distribution. The statistical mean and statistical standard deviation of the background noise level are updated with new normal operating data, and the preset detection threshold is dynamically adjusted.

[0037] In practical implementation, a directed edge is established between two nodes when a measurable cross-correlation peak exists between them and the peak amplitude exceeds a preset detection threshold. The direction of the directed edge points from the node serving as the reference signal in the cross-correlation operation to the node serving as the lag signal. The weight of the directed edge is equal to the corresponding time delay offset. If the first signal sequence is the reference signal and the second signal sequence is the lag signal, the direction of the edge points from the node corresponding to the first signal sequence to the node corresponding to the second signal sequence. The weight of the directed edge is taken as the lag time value corresponding to the cross-correlation peak position, which is obtained by multiplying the lag amount corresponding to the cross-correlation peak position by the sampling interval of the analog-to-digital converter. The lag time values ​​corresponding to the time delay offset sequences correspond one-to-one with the weight values ​​of the directed edges.

[0038] If the peak cross-correlation amplitude between two nodes is not greater than a preset detection threshold, no edge is established between the two nodes. For two nodes that do not have an edge established, the connection relationship between the two nodes is not recorded in the causal relationship graph.

[0039] It is understandable that, among all pairwise node pairs, only those with a cross-correlation peak amplitude greater than a preset detection threshold are established with directed edges. Node pairs with unmeasurable cross-correlation peaks or cross-correlation peak amplitudes not greater than the preset detection threshold are not established with directed edges. The existence of edges between nodes is determined by comparing the cross-correlation peak amplitude with the preset detection threshold. The edge weight is equal to the corresponding element in the time delay offset sequence. The causal relationship graph contains four nodes and several directed edges; the number of directed edges is determined by the number of node pairs that meet the conditions.

[0040] In practice, for each edge in the causal relationship graph connecting two physical field signals, the local similarity in signal morphology is calculated, and fusion weights are assigned according to the signal amplitude ratio. Signal segments corresponding to the two physical field signals are extracted along the time axis using a sliding window method, and normalization is performed on the two signal segments within each window.

[0041] In some embodiments, the length of the sliding window is determined based on the sampling rates of the two physical field signals. Both physical field signals originate from a sequence of signal values ​​after analog-to-digital conversion, and both signals have the same sampling rate, determined by the sampling rate of the analog-to-digital converter. The sliding window length must cover at least one dominant frequency cycle, which is the reciprocal of the dominant frequency component of the vibration signal. The sliding window length is set to the integer value obtained by rounding up the product of the analog-to-digital converter's sampling rate and the dominant frequency cycle. The sliding window step size is set to half the sliding window length, and there is an overlapping interval between adjacent sliding windows.

[0042] In practice, normalization is performed on the two signal segments within each sliding window. The normalization process uses the min-max normalization method. For the first signal segment, the minimum amplitude within the first signal segment is subtracted from the amplitude of all sampled points within that segment, and then divided by the difference between the maximum and minimum amplitudes to obtain the first normalized signal segment. For the second signal segment, the minimum amplitude within the second signal segment is subtracted from the amplitude of all sampled points within that segment, and then divided by the difference between the maximum and minimum amplitudes to obtain the second normalized signal segment. The amplitude range of the sampled points after normalization is from zero to one.

[0043] In some embodiments, the Euclidean distance between the two normalized signal segments is calculated. The Euclidean distance is determined by the following formula: in, It represents the Euclidean distance between two normalized signal segments, in normalized amplitude units, and its value is not less than zero. This represents the length of the sliding window, in units of sampling points, and its value range is positive integers. The sliding window length is set to the integer value obtained by rounding up the product of the analog-to-digital converter's sampling rate, the main frequency period, and the threshold coefficient. The threshold coefficient is set to 1 because the sliding window length covers at least one main frequency period, and the main frequency period is the reciprocal of the main frequency component of the vibration signal. Indicates the first normalized signal segment. The amplitude of each sampling point ranges from zero to one. This indicates the second normalized signal segment. The amplitude of each sampling point ranges from zero to one. This represents the sampling point number, with a value range of one to... .

[0044] In practice, the reciprocal of the Euclidean distance is used as the local similarity of the sliding window. The local similarity is determined by the reciprocal of the Euclidean distance corresponding to the sliding window. When the Euclidean distance is zero, the local similarity value is positive infinity. To avoid overflow of the local similarity value, a preset large value is set when the Euclidean distance is zero. The preset large value is the square of the sliding window length.

[0045] In some embodiments, the average of the local similarities of all windows is taken as the overall morphological similarity between the two physical field signals connected by the edge. The local similarity sequence of the sliding windows is arranged in chronological order, and the sum of the local similarities of all sliding windows is divided by the number of sliding windows to obtain the overall morphological similarity.

[0046] In practice, the overall morphological similarity is used as the base for fusion weight allocation, and fusion weights are assigned according to the signal amplitude ratio. In each edge connecting two physical field signals, the amplitude ratio of the physical field signal pointing to the target node is determined by the target node's own original signal amplitude sequence. The signal amplitude ratio is the ratio of the root mean square value of the target node's own original signal amplitude sequence to the sum of the root mean square values ​​of the original signal amplitude sequences of the two physical field signals. The fusion weight corresponding to each edge is equal to the overall morphological similarity multiplied by the signal amplitude ratio.

[0047] In some embodiments, the fusion weight corresponding to each edge in the causal relationship graph is multiplied by the physical field signal amplitude sequence of the target node it points to, to obtain the weighted signal component contributed by each edge. Each edge corresponds to a fusion weight, and the fusion weight is multiplied by the physical field signal amplitude sequence of the target node it points to to obtain the weighted signal component. The physical field signal amplitude sequence of the target node is the signal amplitude sequence collected by the sensor corresponding to the target node and aligned to the time domain phase.

[0048] In practice, all weighted signal components pointing to the same target node are summed with the original signal amplitude sequence of that target node to obtain the fused signal amplitude sequence of that target node. The original signal amplitude sequence of the target node is the signal segment extracted from the physical field signal corresponding to that target node after phase alignment in the time domain. For target nodes without edge pointing to them, the fused signal amplitude sequence of the target node is equal to the original signal amplitude sequence of the target node.

[0049] In some embodiments, the fused signal amplitude sequences of all target nodes are concatenated into a fused state feature vector according to their node numbers. The node numbering order is: vibration sensor channel, thermocouple sensor channel, ultrasonic probe channel, and oil spectrometer channel. The node number corresponding to the vibration sensor channel is one, the node number corresponding to the thermocouple sensor channel is two, the node number corresponding to the ultrasonic probe channel is three, and the node number corresponding to the oil spectrometer channel is four. The dimension of the fused state feature vector is the sum of the lengths of the fused signal amplitude sequences of the four target nodes, and the fused state feature vectors are arranged in the order of node numbers one to four.

[0050] In specific implementation, please refer to Figure 2 The fusion state feature vectors of the N consecutive time windows preceding the current time window are extracted to form a state transition matrix. The fusion state feature vectors of the N consecutive time windows are calculated by the method described in the above embodiment, and each fusion state feature vector corresponds to the fusion state of the monitoring point within a time window.

[0051] In some embodiments, the fused state feature vectors of N consecutive time windows are arranged in chronological order. The earliest fused state feature vector is designated as the first fused state feature vector, the second earliest as the second fused state feature vector, and so on, until the latest fused state feature vector is designated as the Nth fused state feature vector. The value of N is a positive integer greater than or equal to 2. The value of N is set according to the backtracking duration of the anomaly evolution at the monitoring point. When it is necessary to cover the backtracking length of twelve time windows, N is set to 12. The length of the time window is equal to an integer multiple of the unit's rotation cycle, and the twelve time windows cover the evolution process corresponding to twelve rotation cycles.

[0052] In practice, the fused state feature vectors of every two adjacent time windows within N consecutive time windows form a transition pair. For each adjacent (r+1)th fused state feature vector and the rth fused state feature vector, the (r+1)th fused state feature vector and the rth fused state feature vector are subtracted element-wise to obtain the transition pair vector, which serves as the rth column of the state transition matrix. The value of r ranges from one to N-1. All transition pair vectors are arranged in chronological order to construct the state transition matrix. The number of rows in the state transition matrix is ​​equal to the dimension of the fused state feature vectors, and the number of columns in the state transition matrix is ​​N-1.

[0053] In some embodiments, the dimension of the fused state feature vector is , The state transition matrix is ​​determined by the sum of the lengths of the fused signal amplitude sequences from the four target nodes. The number of rows in the state transition matrix is... The state transition matrix has N-1 columns.

[0054] Optionally, eigenvalue decomposition is performed on the state transition matrix. Eigenvalue decomposition uses either power iteration or QR decomposition to calculate the real parts of the eigenvalues. When using QR decomposition, the state transition matrix is ​​iterated using QR decomposition. The iteration process includes the following operations: performing QR decomposition on the state transition matrix, decomposing it into the product of an orthogonal matrix and an upper triangular matrix; then, exchanging the multiplication order of the orthogonal matrix and the upper triangular matrix to obtain the update matrix; repeating QR decomposition and multiplication exchange until the absolute values ​​of all elements below the diagonal of the update matrix are less than a preset convergence threshold. The diagonal elements of the update matrix are then the complete eigenvalues ​​of the state transition matrix. When using power iteration, an initial vector of all ones is selected. The product of the state transition matrix and the initial vector is calculated and normalized. This iteration is repeated until the Euclidean distance between adjacent iteration vectors is less than a preset convergence threshold, obtaining the eigenvalue with the largest absolute value and its corresponding eigenvector. Then, the remaining eigenvalues ​​are calculated using a shrinking transformation until all eigenvalues ​​are obtained.

[0055] In practical implementation, the eigenvalue decomposition of the state transition matrix is ​​expressed as: in, Represents the state transition matrix; Represents the right eigenvector matrix; Represents an eigenvalue diagonal matrix. The diagonal elements are the eigenvalues ​​of the state transition matrix; This represents the inverse of the right eigenvector matrix. The eigenvalues ​​of the state transition matrix include real eigenvalues ​​and complex eigenvalues.

[0056] In practice, the real part of each eigenvalue is extracted to form a sequence of real parts. This is then applied to the diagonal matrix of eigenvalues. For each diagonal element in the array, take the real part of the diagonal element and write the real parts into the real part sequence according to the diagonal element order. When the eigenvalue is a real number, the real part is the eigenvalue itself; when the eigenvalue is a complex number, the real part is the real part of the complex number.

[0057] In some embodiments, each real part value in the real part sequence is scanned. When at least one real part value is greater than zero, the signal evolution process of the monitoring point is determined to have a divergent trend, and the monitoring point is marked as an anomaly warning source. The scanning process involves comparing the magnitude of each real part value in the real part sequence with zero one by one. If there is a real part value greater than zero, the monitoring point is immediately determined to have a divergent trend and marked as an anomaly warning source. If all real part values ​​in the real part sequence are not greater than zero, the monitoring point is not marked as an anomaly warning source.

[0058] It can be understood that a positive real part of the eigenvalues ​​of the state transition matrix indicates that there is a divergence direction in the signal evolution process. The state transition matrix is ​​composed of the difference vector of the fused state eigenvectors of adjacent time windows. A positive real part indicates that the difference of the fused state eigenvectors of adjacent time windows increases with time.

[0059] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for intelligent inspection of hydropower plants using multimodal data fusion, characterized in that, The method, based on the eigenmode reconstruction and causal tracing chain construction of physical field signals, performs the following steps: Acquire raw signal sequences from multiple physical sensors within the same time window, including vibration signals, temperature signals, ultrasonic signals, and oil spectrum signals; Using the dominant frequency component of the vibration signal as the reference axis, the temperature signal, ultrasonic signal and oil spectrum signal are time-domain phase aligned with the vibration signal to generate a time delay offset sequence between the physical fields. Based on the time delay offset sequence, a causal influence relationship diagram of multi-physics field signals on the time axis is constructed. Each node in the causal influence relationship diagram represents a sampling channel of different physical fields, and each edge represents the time delay offset between two channel signals. For each edge in the causal relationship graph connecting two physical field signals, calculate the local similarity of their signal morphology and allocate fusion weights according to the signal amplitude ratio. The fusion weights are multiplied by the corresponding physical field signals and then superimposed by channel to obtain the fusion state feature vector of the monitoring points within the time window; Extract the fused state feature vectors of N consecutive time windows before the current time window to form a state transition matrix. Calculate the sequence of real parts of the eigenvalues ​​of the state transition matrix and determine whether there are any positive real parts. If there are positive real parts, mark the monitoring point as an abnormal early warning source point.

2. The intelligent inspection method for multimodal data fusion in hydropower plants according to claim 1, characterized in that, Acquire raw signal sequences from multiple physical sensors within the same time window, including vibration signals, temperature signals, ultrasonic signals, and oil spectrum signals, specifically: Vibration sensors, thermocouple sensors, ultrasonic probe transmitter-receiver pairs, and online oil spectrometers are deployed at key locations of each unit in the hydropower plant. All sensors are triggered to start collecting data at the same time using a unified sampling clock, and the data collection duration is equal to an integer multiple of the unit's rotation cycle. The raw signals output by each sensor are converted from analog to digital and then packaged into data frames. Each frame contains a sensor identifier, a timestamp, and a sequence of signal values.

3. The intelligent inspection method for multimodal data fusion in hydropower plants according to claim 2, characterized in that, Using the dominant frequency component of the vibration signal as the reference axis, the temperature signal, ultrasonic signal, and oil spectrum signal are time-domain phase aligned with the vibration signal to generate a time delay offset sequence between the various physical fields, specifically: Perform autocorrelation calculation on the vibration signal, extract the period value corresponding to the peak value of the autocorrelation function, and define the peak frequency as the dominant frequency component; Cross-correlation operations are performed on the temperature signal, ultrasonic signal, and oil spectrum signal with the vibration signal respectively, and the hysteresis time value corresponding to the peak value of each cross-correlation function is extracted as the initial time delay offset of each physical field relative to the vibration signal. After sequentially shifting the vibration signal along the time axis by the initial time delay offset corresponding to each physical field, signal segments within the overlapping time period are re-extracted from the signals of each physical field to complete the time-domain phase alignment.

4. The intelligent inspection method for multimodal data fusion in hydropower plants according to claim 3, characterized in that, Based on the time delay offset sequence, a causal influence diagram of multi-physics field signals on the time axis is constructed. Each node in the causal influence diagram represents a sampling channel of a different physical field, and each edge represents the time delay offset between two signal channels. Specifically: The vibration sensor channel, thermocouple sensor channel, ultrasonic probe channel, and oil spectrometer channel are respectively set as four nodes in the causal relationship diagram; Under the condition that there is a measurable cross-correlation peak between two nodes and the peak amplitude is greater than the preset detection threshold, a directed edge is established between the two nodes. The direction of the edge is from the node that serves as the reference signal in the cross-correlation operation to the node that serves as the hysteresis signal. The weight value of the edge is equal to the corresponding time delay offset. If the peak cross-correlation value between two nodes is not greater than the preset detection threshold, then no edge is established between the two nodes.

5. The intelligent inspection method for multimodal data fusion in hydropower plants according to claim 4, characterized in that, For each edge connecting two physical field signals in the causal relationship graph, the local similarity in signal morphology is calculated, and fusion weights are assigned according to the signal amplitude ratio, specifically as follows: The signal segments of two physical field signals at corresponding moments are extracted along the time axis using a sliding window method, and normalization is performed on the two signal segments in each window. Calculate the Euclidean distance between the two normalized signal segments, and take the reciprocal of the Euclidean distance as the local similarity of the window; The average of the local similarities of all windows is taken as the overall morphological similarity between the two physical field signals connected by the edge, and the overall morphological similarity is used as the base for the fusion weight allocation.

6. The intelligent inspection method for multimodal data fusion in hydropower plants according to claim 5, characterized in that, The fusion weights are multiplied by the corresponding physical field signals and then superimposed by channel to obtain the fusion state feature vector of the monitoring points within the time window, specifically: Multiply the fusion weights corresponding to all edges in the causal relationship graph by the physical field signal amplitude sequence of the target node they point to, to obtain the weighted signal components contributed by each edge. All weighted signal components pointing to the same target node are summed with the original signal amplitude sequence of the target node itself to obtain the fused signal amplitude sequence of the target node. The fused signal amplitude sequences of all target nodes are concatenated in node number order to form a fused state feature vector.

7. The intelligent inspection method for multimodal data fusion in hydropower plants according to claim 6, characterized in that, Extract the fused state feature vectors of N consecutive time windows preceding the current time window to form a state transition matrix. Calculate the sequence of real parts of the eigenvalues ​​of the state transition matrix and determine whether there are any positive real parts. If there are positive real parts, mark the monitoring point as an anomaly warning source point. Specifically: The fused state feature vectors of every two adjacent time windows in N consecutive time windows are formed into a transition pair. All transition pairs are arranged in chronological order to construct a state transition matrix. The number of rows in the state transition matrix is ​​the feature vector dimension, and the number of columns is N-1. The state transition matrix is ​​subjected to eigenvalue decomposition to obtain all eigenvalues. The real part of each eigenvalue is extracted to form a sequence of real parts. Scan each real part value in the real part sequence. When at least one real part value is greater than zero, it is determined that the signal evolution process of the monitoring point has a divergent trend, and the monitoring point is marked as an abnormal early warning source point.

8. The intelligent inspection method for multimodal data fusion in hydropower plants according to claim 3, characterized in that, During the extraction of the dominant frequency component of the vibration signal, if the autocorrelation function shows multiple equally spaced peaks, the frequency corresponding to the peak with the largest amplitude is selected as the dominant frequency component.

9. The intelligent inspection method for multimodal data fusion in hydropower plants according to claim 4, characterized in that, The preset detection threshold is dynamically set based on the statistical mean and standard deviation of the background noise level of each physical field signal.

10. The intelligent inspection method for multimodal data fusion in hydropower plants according to claim 7, characterized in that, The eigenvalue decomposition uses the power iteration method or QR decomposition method to calculate the real eigenvalues ​​of the state transition matrix.

Images