Power equipment fault labeling method based on waveform and unmanned aerial vehicle video fusion

Abstract

The application relates to the technical field of power detection, and discloses a power equipment fault labeling method based on waveform and unmanned aerial vehicle video fusion, which comprises the following steps: waveform amplitude sequences and video image sequences are acquired, spectrum residual analysis is carried out, and a preliminary fluctuation sequence is obtained; the main period of the preliminary fluctuation sequence is calculated to obtain a periodic mode distribution; the changes of adjacent frames in the video image sequences are analyzed to obtain an abnormal image feature set, and the abnormal image feature set is matched with the periodic mode distribution in time stamps to obtain an abnormal association mapping; according to the abnormal association mapping, an abnormal association region is screened out; the abnormal duration and change law of the abnormal association region are analyzed to obtain a cluster state vector, and a preliminary fault type library is inquired to obtain an abnormal classification result; and the abnormal classification result is subjected to signal overheating judgment to subdivide the fault type, and a micro-vibration fault type is obtained. The method can improve the detection precision and classification refinement capability of micro-vibration faults.

Description

Technical Field

[0001] This invention relates to the field of power detection technology, and in particular to a method for labeling power equipment faults based on waveform and UAV video fusion. Background Technology

[0002] Currently, it is difficult to accurately correlate waveform amplitude fluctuations and changes in the appearance of power equipment in terms of time and space, leading to inaccurate identification of fault location and nature. Waveform data reflects the internal state but changes rapidly, while deformation and color difference information in video are easily affected by lighting and viewing angle; there is a lack of synchronous matching methods between the two. Therefore, a technology that can integrate waveform features and image recognition results is needed to achieve real-time correlation analysis in dynamic environments to support accurate fault location.

[0003] In one existing technology, real-time waveform data of power equipment is acquired using current transformers and voltage transformers. A fixed threshold is set for the waveform amplitude sequence. When the amplitude continuously exceeds the threshold for a preset duration, it is marked as an abnormal moment and a timestamp is recorded. Simultaneously, a fixed-position camera captures video of the equipment's appearance. The grayscale mean is extracted from each frame. If the grayscale mean deviates from the historical mean by more than a set deviation, it is marked as an abnormal frame, and a timestamp for that frame is recorded. The timestamps of the waveform abnormal moments and the video abnormal frames are compared point by point. If the time difference between the two is less than a preset threshold, a fault is determined to exist in that time period, and a fault alarm is output. The entire process involves independent waveform analysis and video analysis, with timestamp alignment performed only in the final stage. This existing technology processes waveform data and video images separately, relying solely on timestamps for linear matching. It ignores the nonlinear offset and local deformation of the two on the time axis, resulting in a mismatch between waveform fluctuations and changes in image appearance.

[0004] Therefore, existing technologies suffer from low accuracy in fault location and type identification, and coarse classification results. Summary of the Invention

[0005] This invention provides a method for labeling power equipment faults based on waveform and UAV video fusion, in order to improve the detection accuracy and classification refinement capability of minor vibration faults.

[0006] Firstly, to address the aforementioned technical problems, this invention provides a method for labeling power equipment faults based on waveform and UAV video fusion, comprising:

[0007] Obtain waveform amplitude sequences and video image sequences;

[0008] The waveform amplitude sequence is preprocessed to obtain abnormal fluctuation records, and the timestamps are aligned with the video image sequence to obtain a preliminary fluctuation sequence;

[0009] Anomaly features are extracted from the initial fluctuation sequence to obtain a fluctuation feature sequence, and the principal period of the fluctuation feature sequence is calculated to obtain a periodic pattern distribution;

[0010] An abnormal image feature set is obtained by analyzing the pixel displacement and color difference changes of adjacent frames in the video image sequence, and an abnormal association mapping is obtained by timestamp matching based on the abnormal image feature set and the periodic pattern distribution.

[0011] Based on the abnormal correlation mapping, feature fusion is performed to obtain a dimension-reduced fusion matrix, and abnormal correlation regions are filtered out using a preset correlation strength threshold.

[0012] Clustering is performed on the abnormal associated regions to obtain initial abnormal clusters. The abnormal duration, fluctuation changes and image changes of the initial abnormal clusters are analyzed to obtain cluster state vectors. Based on the cluster state vectors, a preset preliminary fault type library is queried to obtain the abnormal classification results.

[0013] Based on the anomaly classification results, the fault types are further subdivided to obtain micro-vibration fault types.

[0014] Secondly, the present invention provides a power equipment fault labeling system based on waveform and UAV video fusion, comprising:

[0015] The data acquisition module is used to acquire waveform amplitude sequences and video image sequences;

[0016] The residual analysis module is used to preprocess the waveform amplitude sequence to obtain abnormal fluctuation records and align the timestamps with the video image sequence to obtain a preliminary fluctuation sequence.

[0017] The period calculation module is used to extract abnormal features from the preliminary fluctuation sequence to obtain a fluctuation feature sequence, and to calculate the main period of the fluctuation feature sequence to obtain a periodic pattern distribution.

[0018] An anomaly filtering module is used to perform anomaly frame filtering by analyzing the pixel displacement and color difference changes of adjacent frames in the video image sequence to obtain anomaly image feature set, and to perform timestamp matching based on the anomaly image feature set and the periodic pattern distribution to obtain anomaly association mapping;

[0019] The dimension reduction fusion module is used to perform feature fusion based on the abnormal correlation mapping to obtain a dimension reduction fusion matrix and filter out abnormal correlation regions using a preset correlation strength threshold.

[0020] The preliminary classification module is used to perform clustering on the abnormal associated regions to obtain initial abnormal clusters, analyze the abnormal duration, fluctuation changes and image changes of the initial abnormal clusters to obtain cluster state vectors, and query a preset preliminary fault type library based on the cluster state vectors to obtain abnormal classification results;

[0021] The output module is used to subdivide the fault types based on the anomaly classification results to obtain the micro-vibration fault types.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] (1) This invention extracts abnormal fluctuation records by performing spectral residual analysis on the waveform amplitude sequence and aligns them with the timestamp of the video image sequence. Since spectral residual analysis can capture abnormal fluctuations in the waveform that exceed the linear trend, timestamp alignment solves the problem of linear matching between the waveform and the video on the time axis, thereby realizing the preliminary correlation between waveform fluctuations and image appearance changes, laying the foundation for subsequent accurate positioning, overcoming the shortcomings of existing technologies that rely solely on fixed thresholds to judge waveform abnormalities and independently label video frames, and improving the sensitivity of fault detection.

[0024] (2) This invention uses optical flow to analyze the pixel displacement and grayscale difference of adjacent video frames to screen abnormal frames, and uses dynamic time warping algorithm to timestamp match the abnormal image feature set with the periodic pattern distribution of the waveform. Since optical flow quantifies subtle deformations and dynamic time warping processes the nonlinear time offset of the waveform and video, it achieves a precise spatiotemporal correspondence between the waveform and image features, solves the problem of low fault location accuracy caused by nonlinear time axis offset in the prior art, and improves the location accuracy.

[0025] (3) This invention uses principal component analysis to reduce the dimension of the wave and image features and calculates the Pearson correlation coefficient to obtain the correlation intensity distribution. It then filters the abnormal associated regions based on the threshold, and clusters the abnormal associated regions to obtain the cluster state vector and queries the fault type library. Since the dimension reduction and fusion retains the main feature components, the correlation coefficient quantifies the degree of association between the wave and the image, and the cluster analysis classifies similar abnormal patterns, it realizes the refined classification of micro vibration faults, overcomes the defect of coarse classification results in the existing technology, and improves the classification refinement capability. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the power equipment fault labeling method based on waveform and UAV video fusion provided in the first embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of the power equipment fault labeling system based on waveform and UAV video fusion provided in the second embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Reference Figure 1 The first embodiment of the present invention provides a method for power equipment fault labeling based on waveform and UAV video fusion, including the following steps:

[0030] S11, acquire waveform amplitude sequence and video image sequence;

[0031] S12, preprocess the waveform amplitude sequence to obtain abnormal fluctuation records and align the timestamps with the video image sequence to obtain a preliminary fluctuation sequence;

[0032] S13, extract abnormal features from the preliminary fluctuation sequence to obtain a fluctuation feature sequence, and calculate the main period of the fluctuation feature sequence to obtain a periodic pattern distribution;

[0033] S14, abnormal frame screening is performed by analyzing the pixel displacement changes and color difference changes of adjacent frames in the video image sequence to obtain an abnormal image feature set, and an abnormal association mapping is obtained by timestamp matching based on the abnormal image feature set and the periodic pattern distribution;

[0034] S15, perform feature fusion based on the abnormal correlation mapping to obtain a dimension-reduced fusion matrix and use a preset correlation strength threshold to filter out abnormal correlation regions;

[0035] S16, perform clustering on the abnormal associated region to obtain an initial abnormal cluster, analyze the abnormal duration, fluctuation changes and image changes of the initial abnormal cluster to obtain a cluster state vector, and query a preset preliminary fault type library based on the cluster state vector to obtain an abnormal classification result;

[0036] S17, Based on the anomaly classification results, the fault types are further subdivided to obtain micro-vibration fault types.

[0037] In step S11, the waveform amplitude sequence and video image sequence are obtained.

[0038] Specifically, the process involves acquiring waveform amplitude sequences and video image sequences. The waveform amplitude sequences are derived from high-precision current and voltage transformers installed on the power equipment. Real-time waveform data is continuously acquired at a fixed sampling frequency (e.g., 1000 sampling points per second), with each sampling point recording the current or voltage amplitude at that moment. The video image sequences are acquired by cameras or drones deployed around the equipment at a fixed frame rate (e.g., 30 frames per second), with each frame containing a pixel matrix of the equipment's appearance. The two sequences are synchronized using a unified timestamp system to ensure a correspondence between waveform data and video frames at the same moment. This step provides the raw input data for subsequent spectral residual analysis and image anomaly detection, forming the foundation of the entire method.

[0039] In step S12, the waveform amplitude sequence is preprocessed to obtain abnormal fluctuation records, and the timestamps are aligned with the video image sequence to obtain a preliminary fluctuation sequence, including:

[0040] Based on the waveform amplitude sequence, noise is filtered using a Gaussian filtering algorithm to obtain a clean waveform sequence.

[0041] The pure waveform sequence is time-series segmented according to a preset sliding time window, and the waveform peaks in each window are extracted to obtain a set of waveform peaks;

[0042] Based on the waveform peak set, a spectrum transformation is performed using the Fast Fourier Transform algorithm to obtain a spectrum feature sequence. Then, based on the spectrum feature sequence, a linear fitting is performed using the least squares method to obtain a linear fitting sequence.

[0043] The difference between the spectral feature sequence and the linear fitting sequence is calculated point by point to obtain the fitting residual sequence. Then, time nodes corresponding to fitting residuals greater than a preset fitting residual threshold are selected to identify abnormal time nodes.

[0044] The consecutive abnormal time nodes are merged into intervals to obtain abnormal fluctuation records. The abnormal fluctuation records and the video image sequence are then timestamped to obtain a preliminary fluctuation sequence.

[0045] Specifically, noise is filtered using a Gaussian filtering algorithm based on the waveform amplitude sequence. The Gaussian filter employs a convolution kernel of a preset kernel length, with weights at each position within the kernel following a Gaussian distribution, maximizing the weights at the center and minimizing those at the edges. The convolution kernel is slid along the time axis, and the filtered value at each time point is the sum of the products of the sampled values ​​and their corresponding weights within the kernel's coverage area, divided by the total weights. After traversing all time points, a clean waveform sequence is obtained, which removes high-frequency random noise while preserving the main trend of the waveform. The kernel length of the Gaussian filter is determined based on the sampling frequency of the waveform amplitude sequence. For example, the sampling frequency is divided by fifty and the nearest odd number is taken, with the kernel length not less than five. When the sampling frequency is unknown, the default kernel length is eleven.

[0046] The clean waveform sequence is time-series segmented according to a preset sliding time window. The length and step size of the sliding time window are determined by analyzing waveform data under historical normal operating conditions. A continuous waveform amplitude sequence during fault-free operation is acquired, the autocorrelation function of the sequence is calculated, and the time offset corresponding to the first drop of the autocorrelation coefficient to half of the initial value is found. This offset is used as the initial value of the window length. The historical data is segmented using this initial length, the number of waveform peaks in each window is counted, and the rate of change of the number of peaks between adjacent windows is calculated. The 95th percentile of all rate of change values ​​is taken as the fluctuation tolerance. If the rate of change of the number of peaks in adjacent windows exceeds the fluctuation tolerance, the window length is appropriately shortened; if the number of peaks in each window is too small (e.g., less than three complete cycles), the window length is appropriately increased. After multiple iterations, the minimum length that keeps the rate of change of the number of peaks within the fluctuation tolerance is selected as the final window length.

[0047] The sliding step size is set at 40% of the window length as the base step size. The specific value of the step size is determined through cross-validation. Centered on the base step size, the step size is decreased and increased in increments (e.g., 5% of the window length) to form multiple candidate step sizes. For each candidate step size, historical data is slidably segmented according to that step size, and the Pearson correlation coefficients of all adjacent window peak sequences are calculated. The average of these correlation coefficients is taken. The correlation coefficient threshold is determined by analyzing historical normal waveform data. Another continuous waveform amplitude sequence during fault-free operation of the equipment is obtained, and slidably segmented using the final window length and multiple candidate step sizes. The Pearson correlation coefficients of all adjacent window peak sequences are calculated, and the 95th percentile of these correlation coefficient values ​​is taken as the correlation coefficient threshold. The smallest step size that results in an average correlation coefficient greater than this threshold is selected as the final step size.

[0048] It should be noted that the adjustment of the sliding time window length adopts the following iterative rule: each time the length is shortened or increased by 10% of the current window length; the iteration termination condition is that when the rate of change of the number of peaks in adjacent windows is less than or equal to the fluctuation tolerance, and the number of peaks in each window is greater than or equal to three complete cycles, the adjustment stops; if the above two conditions cannot be met simultaneously after three consecutive adjustments, the window length that minimizes the absolute value of the difference between the rate of change and the fluctuation tolerance is taken as the final length. The step value is fixed at 5% of the window length; the candidate step length is generated within the range of 50% to 150% of the base step length, that is, based on the base step length, the step value is gradually decreased until it reaches 50% of the base step length, and gradually increased until it reaches 150% of the base step length.

[0049] Within each window, a peak detection algorithm is used to extract waveform peaks. The sampling points within the window are traversed; if the amplitude of a point is greater than its left and right adjacent points and greater than the preset minimum peak spacing condition, then that point is recorded as a peak. The preset minimum peak spacing is determined using waveform data from historical normal operating conditions. A continuous waveform amplitude sequence is obtained during fault-free operation of the equipment. After dividing the sequence into identical sliding windows, the time interval between adjacent peaks is counted within each window. The interval values ​​of all windows are merged, and the 50th percentile of these interval values ​​is taken as the initial value of the minimum peak spacing. The 95th percentile is then taken as the upper limit, and the spacing value that minimizes the sum of the peak false detection rate and the peak missed detection rate is selected as the final minimum peak spacing. The peaks of all windows are arranged in chronological order to obtain a set of waveform peaks.

[0050] The peak false detection rate is obtained by using a manually annotated set of historical waveform peaks as a reference true value, matching the peak set detected by the algorithm with the reference true value, and dividing the number of unmatched true peaks by the total number of peaks in the reference true value. The peak false detection rate is obtained by dividing the number of peaks detected by the algorithm that fail to match any reference true value by the total number of peaks detected by the algorithm. The manual annotation uses waveform peak recognition tools, such as MATLAB's findpeaks function, to assist in annotation and is manually reviewed and confirmed.

[0051] Based on the waveform peak set, a spectrum transformation is performed using the Fast Fourier Transform (FFT) algorithm. The FFT converts the time-domain waveform peak sequence into a frequency-domain representation, outputting the complex amplitude corresponding to each frequency point. The modulus of each complex amplitude is then taken to obtain the spectral feature sequence. Next, based on the spectral feature sequence, a linear fit is performed using the least squares method to calculate the best-fit line between frequency and amplitude in the spectral feature sequence, minimizing the sum of the squared perpendicular distances from all data points to the line. After determining the slope and intercept of the fitted line, the fitted values ​​corresponding to each frequency point constitute a linear fit sequence.

[0052] The difference between the spectral feature sequence and the linearly fitted sequence is calculated point by point, that is, the difference between the actual amplitude and the fitted value at the corresponding frequency point, to obtain the fitted residual sequence. The fitted residual threshold is determined by statistical analysis of historical data under normal operating conditions. A large number of spectral feature sequences are collected during the fault-free operation of the equipment, and the fitted residual of each historical sequence is calculated. The 95th percentile of all historical fitted residual values ​​is taken as the fitted residual threshold. Time nodes with residual values ​​greater than the threshold in the fitted residual sequence are selected to obtain abnormal time nodes. Consecutive abnormal time nodes are merged into intervals, that is, adjacent time nodes are grouped into the same interval if the time is continuous, to obtain abnormal fluctuation records.

[0053] The abnormal fluctuation records and video image sequences are timestamped. Each interval in the abnormal fluctuation records contains a start time and an end time, and each frame in the video image sequence is timestamped. The aligned result is output as a preliminary fluctuation sequence, which provides a temporal reference for subsequent periodic analysis and image feature matching.

[0054] In step S13, anomaly feature extraction is performed on the preliminary fluctuation sequence to obtain a fluctuation feature sequence, and the principal period of the fluctuation feature sequence is calculated to obtain a periodic pattern distribution, including:

[0055] Calculate the mean and standard deviation of the amplitude of the preliminary fluctuation sequence within the sliding time window, and extract the extreme points of fluctuations whose amplitude exceeds the mean plus twice the standard deviation to obtain the set of abnormal extreme points;

[0056] Based on the set of abnormal extreme points, envelope feature vectors are extracted using the Hilbert transform algorithm to obtain the envelope feature vector set.

[0057] Calculate the Euclidean distance between each sliding time window in the envelope feature vector set to obtain the sliding distance matrix;

[0058] The slip distance matrix is ​​clustered using a hierarchical clustering algorithm to obtain the fluctuation characteristic sequence;

[0059] Based on the fluctuation characteristic sequence, the principal period is calculated using a pre-constructed autoregressive model to obtain the periodic pattern distribution.

[0060] The process of constructing an autoregressive model includes:

[0061] Historical fluctuation sequences are obtained, the model order is selected using the Akaike Information Criterion, and the coefficients are estimated using the least squares method based on the historical fluctuation sequences and the model order to obtain the autoregressive model coefficients.

[0062] Based on the historical fluctuation sequence, the model order, and the autoregressive model coefficients, the model fit value is calculated, and the difference between the model fit value and the historical fluctuation sequence is calculated to obtain the model residual sequence.

[0063] Based on the model residual sequence, the model validity was verified using the Ljung-Box test method to obtain the white noise test value;

[0064] When the white noise test value is greater than the preset white noise test threshold, the model is deemed valid, and the constructed autoregressive model is obtained.

[0065] When the white noise test value is not greater than the white noise test threshold, the order selection and coefficient estimation are repeated until the white noise test value is greater than the white noise test threshold, thus obtaining the autoregressive model.

[0066] Specifically, anomaly features are extracted from the initial fluctuation sequence to obtain a fluctuation feature sequence, and the principal period is calculated to obtain a periodic pattern distribution. The mean and standard deviation of the amplitude of the initial fluctuation sequence within each sliding time window are calculated. For each window, the amplitudes of all sampling points are summed and divided by the number of sampling points to obtain the mean; the sum of the squares of the differences between each amplitude and the mean is calculated, divided by the number of sampling points, and the square root is taken to obtain the standard deviation. Then, fluctuation extreme points with amplitudes exceeding the mean plus twice the standard deviation are extracted. All sampling points within the window are traversed, and if the amplitude of a point is greater than the mean plus twice the standard deviation, the point is marked as a fluctuation extreme point. The extreme points of all windows are arranged in chronological order to obtain a set of abnormal extreme points.

[0067] Based on the set of outlier extreme points, envelope feature vectors are extracted using the Hilbert transform algorithm. The Hilbert transform performs frequency domain processing on the extreme point sequence to construct an analytic signal, calculating the instantaneous amplitude and phase at each time point. The instantaneous amplitude sequence within each window is used as the envelope feature of that window. Each window's envelope feature contains values ​​from multiple time points, which need to be compressed into a feature vector of fixed length. The compression length is determined as follows: a large number of window envelope feature sequences are collected under normal operating conditions. Principal component analysis is performed on the envelope features of each window to calculate the cumulative contribution rate of the principal components. The number of principal components required to achieve a cumulative contribution rate of 95% is selected as the initial value for the compression length. Then, clustering experiments are conducted near the initial value (e.g., adding or subtracting 3). Using the envelope feature vector at this length as input, a hierarchical clustering algorithm is run to calculate the silhouette coefficients of the clustering results. The length that maximizes the silhouette coefficient is selected as the final compression length. If the silhouette coefficients corresponding to multiple lengths are similar, the minimum value is taken to reduce computational complexity. After determining the compression length, the envelope features of each window are downsampled or statistically extracted (such as mean, variance, peak value, etc.) to obtain a fixed-length envelope feature vector. The feature vectors of all windows are arranged in chronological order to obtain the envelope feature vector set.

[0068] It should be noted that the compression method of the envelope feature vector is fixed to use equal-interval downsampling. Specifically, the envelope feature sequence within each sliding time window is evenly divided into ten sub-intervals, and the maximum value in each sub-interval is taken as the representative value of that sub-interval, which is then combined to form a ten-dimensional envelope feature vector. When the number of sampling points in the window is less than ten, linear interpolation is used to expand the sequence to ten points before downsampling.

[0069] Calculate the Euclidean distance between each sliding time window in the envelope feature vector set. For any two window feature vectors, subtract the corresponding elements, square the result, sum all the squares, and take the square root to obtain the distance between the two windows. Perform the above calculation for all window pairs to obtain a square matrix, which is the sliding distance matrix.

[0070] The sliding distance matrix is ​​clustered using a hierarchical clustering algorithm. Hierarchical clustering first treats each window as an independent cluster, then calculates the inter-cluster distance (using the average chain method, i.e., the average distance between all windows in two clusters). The two closest clusters are merged at each step, and this merging process is repeated until all windows are merged into one cluster or a preset number of clusters is reached. Based on the clustering results, windows belonging to the same cluster are arranged in chronological order to obtain a fluctuating feature sequence, where each element represents a feature of a cluster (such as the average of the envelope features of all windows within the cluster).

[0071] Based on the fluctuation characteristic sequence, the principal period is calculated using a pre-constructed autoregressive model. The autoregressive model assumes a linear relationship between the current value and several previous historical values. The fluctuation characteristic sequence is input into the model, which outputs the power spectral density of the sequence. The frequency component with the highest power spectral density is found, and its reciprocal is the principal period, outputting a periodic pattern distribution.

[0072] The construction process of the autoregressive model is as follows: First, a historical fluctuation sequence is obtained. This sequence is extracted from the historical waveform amplitude sequence during the fault-free operation of power equipment, with a extraction duration of no less than 100 complete cycles, and the sampling frequency is consistent with the real-time waveform data. Then, a stationarity test is performed. If the sequence is non-stationary, first-order differencing is performed until the sequence becomes stationary. Next, the stationary sequence is normalized, mapping the numerical range to the zero-to-one interval to eliminate the influence of dimensions. Since normalization is a linear transformation, it does not affect the periodicity of the sequence. The model order is selected using the Akaike Information Criterion. The Akaike Information Criterion is calculated as follows: for each candidate order, the model coefficients are estimated using the least squares method, and the residual variance is calculated. The criterion value is obtained by calculating the ratio of the sum of squared residuals to the sequence length, taking the natural logarithm of this ratio, multiplying it by the sequence length, and finally adding twice the order. The order with the smallest criterion value is selected as the model order. Based on the historical fluctuation sequence and the selected model order, the autoregressive coefficients are estimated using the least squares method. The historical sequence is constructed as a system of linear equations, and the coefficients that minimize the sum of squared residuals are solved to obtain the autoregressive model coefficients. Based on the historical fluctuation sequence, model order, and autoregressive model coefficients, the model fit value is calculated. For each time point, the fitted value is obtained by multiplying the previous historical values ​​of the specified order by their corresponding coefficients and summing the results. The historical fluctuation sequence is subtracted from the fitted value point by point to obtain the model residual sequence. The model validity is verified using the Ljung-Box test. The autocorrelation function of the residual sequence is calculated, and a test statistic is constructed to obtain the white noise test value. When the white noise test value is greater than a preset white noise test threshold, the residual sequence is considered white noise, the model is valid, and the constructed autoregressive model is obtained. The white noise test threshold is taken as the 95th percentile of the chi-square distribution. When the white noise test value is not greater than this threshold, the order selection and coefficient estimation are repeated until the condition is met, resulting in the autoregressive model. This model is used for principal period calculation, enabling the periodic pattern distribution to accurately reflect the inherent periodic characteristics of fluctuations.

[0073] In step S14, abnormal frame screening is performed by analyzing the pixel displacement and color difference changes of adjacent frames in the video image sequence to obtain an abnormal image feature set. Then, timestamp matching is performed based on the abnormal image feature set and the periodic pattern distribution to obtain an abnormal association mapping, including:

[0074] The pixel displacement vectors of adjacent frames in the video image sequence are calculated using the Lucas-Kanade optical flow method to obtain the pixel displacement sequence, and the grayscale difference between adjacent frames in the video image sequence is calculated to obtain the color difference change sequence.

[0075] The pixel displacement sequence and the color difference change sequence are combined in chronological order to obtain a frame feature vector sequence.

[0076] Calculate the Euclidean distance between adjacent frame feature vectors in the frame feature vector sequence to obtain the inter-frame distance sequence;

[0077] Video frames whose inter-frame distance is greater than a preset inter-frame distance threshold in the inter-frame distance sequence are selected to obtain an abnormal image feature set;

[0078] Based on the abnormal image feature set and the periodic pattern distribution, timestamp matching is performed using a dynamic time warping algorithm to obtain an abnormal association mapping.

[0079] Specifically, anomaly frame screening is performed by analyzing pixel displacement and color difference changes in adjacent frames of a video image sequence to obtain an anomaly image feature set. Anomaly association mapping is then obtained by timestamp matching based on the anomaly image feature set and periodic pattern distribution. The Lucas-Kanade optical flow method is used to calculate the pixel displacement vectors of adjacent frames in the video image sequence. The Lucas-Kanade optical flow method assumes constant pixel brightness and consistent local motion between adjacent frames. For each pixel in each frame, the optical flow equation is solved within a local window to obtain the horizontal and vertical displacement components of that point. The displacement components of all pixels are organized sequentially by frame, resulting in a displacement vector for each frame (e.g., taking the mean or median of the total frame displacements), thus forming a pixel displacement sequence. Simultaneously, the grayscale difference between adjacent frames in the video image sequence is calculated. The grayscale values ​​of corresponding pixel positions in two adjacent frames are subtracted, the absolute value is taken, and then the average of all pixel differences in the entire frame is calculated to obtain the color difference change value for that frame. The color difference values ​​of all frames are arranged in chronological order to obtain a color difference change sequence.

[0080] Based on the pixel displacement sequence and the color difference change sequence, they are combined in chronological order to obtain the frame feature vector sequence. For each frame, its corresponding pixel displacement value and color difference change value are combined into a two-dimensional feature vector. The feature vectors of all frames are arranged in chronological order to form the frame feature vector sequence.

[0081] Calculate the Euclidean distance between feature vectors of adjacent frames in the frame feature vector sequence. For feature vectors of two adjacent frames, calculate the sum of squares of the differences in corresponding dimensions, and then take the square root to obtain the inter-frame distance. Arrange the distance values ​​of all adjacent frames in chronological order to obtain the inter-frame distance sequence.

[0082] Video frames whose inter-frame distances exceed a preset inter-frame distance threshold are selected from the inter-frame distance sequence to obtain an abnormal image feature set. The inter-frame distance threshold is determined statistically from video data during historical normal operation. A large number of video sequences are collected during fault-free operation of the equipment, and the inter-frame distance of each historical sequence is calculated using the same process. The 95th percentile of all historical inter-frame distance values ​​is taken as the inter-frame distance threshold. When the inter-frame distance of a frame exceeds this threshold, the frame is marked as an abnormal frame. The frame number, timestamp, and corresponding frame feature vector of all abnormal frames are collected to form the abnormal image feature set.

[0083] Based on the abnormal image feature set and periodic pattern distribution, a dynamic time warping algorithm is used for timestamp matching to obtain an anomaly association mapping. The dynamic time warping algorithm is used to calculate the similarity between two time series and find the optimal alignment path. The timestamp of each abnormal frame in the abnormal image feature set is taken as the first time series, and the timestamp corresponding to each periodic peak in the periodic pattern distribution is taken as the second time series. The dynamic time warping algorithm constructs a distance matrix, calculates the cumulative distance matrix, and finds the path with the minimum cumulative distance from the top left to the bottom right, thereby achieving non-linear alignment of the first and second time series on the time axis. After alignment, each abnormal frame is associated with one or more periodic time points, forming matching point pairs. These matching point pairs are organized chronologically to obtain the anomaly association mapping. This mapping records the spatiotemporal correspondence between waveform periodic fluctuations and abnormal video frames, providing a basis for subsequent feature fusion and region localization.

[0084] In step S15, feature fusion is performed based on the abnormal correlation mapping to obtain a dimensionality-reduced fusion matrix, and abnormal correlation regions are filtered out using a preset correlation strength threshold, including:

[0085] Calculate the time difference between each pair of matching points in the abnormal association mapping to obtain a time deviation sequence, and filter out the matching point pairs in the time deviation sequence whose time deviation is less than a preset time deviation threshold to obtain valid matching point pairs;

[0086] The fluctuation features corresponding to the effective matching point pairs are extracted from the periodic pattern distribution to obtain the interval fluctuation feature set, and the abnormal image features corresponding to the effective matching point pairs are extracted from the abnormal image feature set to obtain the interval image feature set;

[0087] Based on the interval fluctuation feature set and the interval image feature set, a dimension reduction and fusion matrix is ​​obtained by performing dimensionality reduction and fusion using the principal component analysis algorithm;

[0088] Calculate the Pearson correlation coefficient between the fluctuation features and image features at each position in the dimensionality reduction fusion matrix to obtain the correlation intensity distribution;

[0089] The regions corresponding to correlation intensities greater than a preset correlation intensity threshold in the correlation intensity distribution are selected to obtain abnormal correlation regions.

[0090] Specifically, a dimensionality-reduced fusion matrix is ​​obtained by fusing wave features and image features based on the anomaly correlation mapping. Anomaly-related regions are then selected from this matrix using a preset correlation strength threshold. The time difference between each matching point pair in the anomaly correlation mapping is calculated. The anomaly correlation mapping contains multiple matching point pairs, each consisting of a time point in the waveform's periodic distribution and a timestamp from the anomaly image feature set. For each point pair, the absolute value of the subtraction of the two timestamps is taken to obtain the time difference. The time difference values ​​of all point pairs are arranged sequentially to obtain a time deviation sequence. The time deviation threshold is determined through statistical analysis of historical data under normal operating conditions. Multiple historical matching point pairs are collected during fault-free operation of the equipment, and the time deviation of each pair is calculated. The 95th percentile of all historical time deviation values ​​is taken as the time deviation threshold. Matching point pairs with deviation values ​​less than this threshold are selected from the time deviation sequence to obtain valid matching point pairs. These point pairs represent anomaly events where the waveform and image are highly synchronized in time.

[0091] The fluctuation features corresponding to valid matching point pairs are extracted from the periodic pattern distribution. Each periodic peak in the periodic pattern distribution corresponds to a time point and fluctuation feature parameters (such as amplitude envelope, frequency, etc.) near that time point. Based on the waveform time points in the valid matching point pairs, the fluctuation features corresponding to the time points are extracted from the periodic pattern distribution, and all extracted features are arranged in chronological order to obtain the interval fluctuation feature set. Simultaneously, the abnormal image features corresponding to valid matching point pairs are extracted from the abnormal image feature set. Each abnormal frame in the abnormal image feature set contains a frame feature vector (pixel displacement and color difference). Based on the video timestamps in the valid matching point pairs, the feature vectors of the corresponding frames are extracted from the abnormal image feature set to obtain the interval image feature set.

[0092] Based on the interval fluctuation feature set and the interval image feature set, dimensionality reduction and fusion are performed using principal component analysis (PCA). PCA first merges the two feature sets into a high-dimensional matrix, where each row corresponds to a matching point pair and each column corresponds to a feature dimension (including fluctuation and image features). Then, the covariance matrix of this matrix is ​​calculated, and the eigenvalues ​​and eigenvectors of the covariance matrix are solved. The eigenvalues ​​are sorted from largest to smallest, and the top few eigenvectors with a cumulative contribution rate reaching 95% are selected. The original high-dimensional matrix is ​​projected onto the low-dimensional space spanned by these eigenvectors, resulting in a dimensionality-reduced fusion matrix. This matrix retains the main information of the original features while reducing the data dimensionality.

[0093] Calculate the Pearson correlation coefficient between the fluctuation features and image features at each location in the dimensionality-reduced fusion matrix. For each row in the dimensionality-reduced fusion matrix (corresponding to a matching point pair), treat the fluctuation feature part and the image feature part as two variable sequences respectively, and calculate the Pearson correlation coefficient. First, calculate the mean of the two sequences, and then calculate the sum of the products of the deviations of each value from its mean, divided by the square root of the sum of squared deviations. Organize the correlation coefficients of all matching point pairs according to spatial location (such as image coordinates or time intervals) to obtain the correlation intensity distribution.

[0094] The system identifies regions within the correlation intensity distribution where the correlation intensity exceeds a preset threshold. This threshold is determined through statistical analysis of historical data from normal operation. A large number of historical correlation intensity distributions are collected during fault-free equipment operation, and the maximum correlation intensity value in each distribution is calculated. The 95th percentile of all historical maximum values ​​is then used as the correlation intensity threshold. Within the correlation intensity distribution, regions with intensity values ​​exceeding this threshold (e.g., pixel coordinate ranges or time intervals in an image) are identified and marked as anomalous correlation regions. These regions indicate where waveform anomalies and image anomalies highly overlap in time and space, providing precise location data for subsequent fault classification.

[0095] In step S16, clustering is performed on the abnormal associated regions to obtain initial abnormal clusters. The abnormal duration, fluctuation changes, and image changes of the initial abnormal clusters are analyzed to obtain cluster state vectors. Based on the cluster state vectors, a preset preliminary fault type library is queried to obtain abnormal classification results, including:

[0096] The fluctuation characteristics and image features within the abnormal correlation region are clustered using the K-means clustering algorithm to obtain initial abnormal clusters;

[0097] Calculate the mean interval duration of each cluster within the initial abnormal cluster to obtain the average cluster duration;

[0098] By using the least squares method, linear fitting is performed on the fluctuation characteristics and image characteristics of all intervals in each cluster within the initial anomaly cluster to obtain the fluctuation fitting line and the image fitting line. The slopes of the fluctuation fitting line and the image fitting line are then extracted to obtain the fluctuation change slope and the image change slope.

[0099] The average duration of the cluster, the slope of the fluctuation change, and the slope of the image change are combined into a cluster state vector;

[0100] Based on the cluster state vector, fault mode matching is performed using a preset preliminary fault type library to obtain anomaly classification results.

[0101] Specifically, clustering is performed on the anomaly-related regions to obtain initial anomaly clusters. The duration of the anomaly, fluctuation changes, and image changes of each cluster are analyzed to obtain a cluster state vector. Then, based on the cluster state vector, a pre-defined preliminary fault type library is queried to obtain the anomaly classification result. The K-means clustering algorithm is used to perform cluster analysis on the fluctuation characteristics and image features within the anomaly-related regions. The anomaly-related regions consist of multiple spatial locations or time intervals, each corresponding to a set of fluctuation characteristics (such as frequency change rate, amplitude envelope, etc.) and image features (such as deformation amplitude, color difference, etc.). K-means clustering requires pre-setting the number of clusters and the maximum number of iterations. The number of clusters is determined using the silhouette coefficient method. For a candidate cluster size range (e.g., 2 to 10), the K-means algorithm is run and the silhouette coefficient of each sample is calculated. The cluster with the largest average silhouette coefficient is taken as the final cluster size. The method for determining the maximum number of iterations is as follows: Collect a large number of historical feature samples from the device under normal operating conditions. Using the final determined number of clusters, set different candidate values ​​for the number of iterations (e.g., 20, 50, 100, 200). Run the K-means algorithm for each candidate value, recording the change in the objective function (sum of squared errors within clusters) after each iteration. When the change in the objective function is less than a preset threshold (e.g., one-thousandth), the algorithm is considered to have converged. Take the maximum number of iterations required for the algorithm to converge from all historical samples, and add 20% as a safety margin to obtain the maximum number of iterations. If this value exceeds a preset upper limit (e.g., 200), the upper limit is used. The specific process of the K-means algorithm is as follows: Randomly initialize a specified number of cluster centers, assign each sample to the nearest cluster center, and then recalculate the mean of all samples within each cluster as the new cluster center. Repeat the assignment and update steps until the maximum number of iterations is reached or the cluster centers no longer change. After clustering, each sample is assigned a cluster label, and all samples are grouped according to the cluster labels to obtain the initial abnormal clusters.

[0102] Calculate the mean duration of intervals for each cluster within the initial anomaly cluster. Each anomaly-associated region corresponds to a time interval (start and end time), and the interval duration is the end time minus the start time. For each cluster, sum the durations of all regions within that cluster and divide by the number of regions to obtain the cluster average duration. Simultaneously, perform linear fitting on the fluctuation characteristics and image features of all intervals within each cluster using the least squares method. Using the time sequence of the intervals as the independent variable and the fluctuation characteristic value (e.g., the rate of change of vibration frequency) as the dependent variable, find the best-fit line that minimizes the sum of the squared vertical distances from all data points to the line. The slope of the fitted line represents the rate of change of the fluctuation characteristic over time. Similarly, perform the same linear fitting on the image features (e.g., deformation amplitude) to obtain the image change slope. Combine the cluster average duration, fluctuation change slope, and image change slope into a three-dimensional vector, i.e., the cluster state vector.

[0103] Based on the cluster state vector, fault mode matching is performed using a pre-defined preliminary fault type library. The preliminary fault type library is a pre-built lookup table where each record contains a feature range for a fault mode, such as a duration of 1.5 to 2.5 seconds, a fluctuation slope of 0.3 to 0.5, an image slope of 0.2 to 0.4, and a corresponding fault type label, such as "bearing wear" or "local overheating." Each component in the cluster state vector is compared with the feature range in the fault type library. If all components fall within the range of a certain record, a match is successful, and the fault type label corresponding to that record is extracted as the anomaly classification result. If no completely matching record is found, the record with the closest Euclidean distance is selected as the output. This step achieves refined classification from spatiotemporally related regions to specific fault types, providing input for subsequent overheating judgment.

[0104] Specifically, the preliminary fault type library collects historical fault event data, with each event including a cluster state vector and a corresponding fault type label. Then, for all cluster state vectors under the same fault type, the mean and standard deviation of each component are calculated, and the mean plus or minus twice the standard deviation is used as the feature range of the fault type. Finally, the fault type label and its feature range are stored in the library. If the number of samples for a certain fault type is less than thirty, the percentile method is used, and the fifth percentile to the ninety-fifth percentile of each component is used as the feature range.

[0105] In step S17, the fault type is further subdivided according to the anomaly classification result to obtain the micro-vibration fault type, including:

[0106] The signal segments corresponding to the anomaly classification results are extracted from the waveform amplitude sequence and the video image sequence, respectively, to obtain waveform signal segments and video signal segments;

[0107] Based on the waveform signal segment, frequency domain transformation is performed using the Fast Fourier Transform algorithm to obtain the micro-vibration characteristics;

[0108] When there are consecutive preset number of grayscale values ​​in the video signal segment that are all greater than a preset grayscale threshold, an overheated area identifier is generated to represent the true overheated area.

[0109] When there are no consecutive frames in the video signal segment whose grayscale values ​​are all greater than the grayscale threshold, a false overheating area identifier is generated.

[0110] Based on the micro-vibration characteristics and the overheated area identification, the micro-vibration fault type is obtained by classifying the faults using pre-built fault mapping rules.

[0111] The process of constructing fault mapping rules includes:

[0112] Acquire historical vibration characteristics, historical overheating indicators, and historical fault types;

[0113] Using the historical vibration characteristics and historical overheating identifiers as inputs and the historical fault type as outputs, the initial mapping rules are generated through the CART decision tree algorithm to obtain the initial rule set.

[0114] Based on the initial rule set, redundant branches are removed using a cost-complexity pruning algorithm to obtain fault mapping rules.

[0115] Specifically, an overheating judgment is performed on the anomaly classification results using a preset grayscale threshold to obtain overheated area identifiers. Based on these identifiers, the fault type is further subdivided to determine the micro-vibration fault type. First, signal segments corresponding to the anomaly classification results are extracted from both the waveform amplitude sequence and the video image sequence. The anomaly classification results contain the time interval of the fault occurrence. Based on this time interval, sampling points corresponding to the time period are extracted from the waveform amplitude sequence to obtain waveform signal segments; simultaneously, continuous frames corresponding to the time period are extracted from the video image sequence to obtain video signal segments.

[0116] Based on the waveform signal segments, frequency domain transformation is performed using the Fast Fourier Transform (FFT) algorithm. The FFT converts the time-domain waveform signal segments into a frequency-domain representation, outputting the complex amplitude corresponding to each frequency point. The magnitude of each complex amplitude is taken to obtain the spectral distribution. The frequency corresponding to the largest amplitude in the spectrum is extracted as the dominant frequency, and the ratio of the amplitude of this dominant frequency to a normal reference value is calculated. The normal reference value is determined by collecting a large number of historical waveform signal segments (e.g., 100 segments) under fault-free normal operating conditions. The same FFT is performed on each signal segment to extract the dominant frequency amplitude of each segment, and the average of all dominant frequency amplitudes is calculated as the initial reference value. Then, the ratio of the dominant frequency amplitude of each segment to the initial reference value is calculated. After removing abnormal segments with ratios exceeding the range of 0.5 to 1.5, the average of the dominant frequency amplitudes of the remaining segments is recalculated, and this average is used as the final normal reference value. This range is determined based on the empirical distribution of dominant frequency amplitude fluctuations under normal operating conditions, and the corresponding normal fluctuation amplitude will not deviate from the average value by more than 50%. If the amount of historical data is insufficient (e.g., less than 30 segments), the equipment's factory calibration value is used as the normal reference value. The main frequency, amplitude ratio, and other frequency domain characteristics (such as spectral energy concentration) are combined to form micro-vibration characteristics.

[0117] For video signal segments, grayscale values ​​are calculated frame by frame. The grayscale threshold is determined by collecting a large number of historical video frames during normal operation of the equipment, calculating the average grayscale value of each frame, and taking the 95th percentile of all historical grayscale values ​​as the grayscale threshold. The preset number of consecutive frames is determined by analyzing the grayscale change characteristics of video frames during normal operation of the equipment and the statistical results of historical fault samples. First, a continuous video sequence (at least 10 minutes in length) is collected during fault-free operation of the equipment, and the average grayscale value of each frame is calculated to obtain a grayscale sequence. Autocorrelation analysis is performed on this grayscale sequence to calculate the curve of the autocorrelation coefficient changing with the frame offset. The frame offset corresponding to the first drop of the autocorrelation coefficient to half of the initial value is found, and this offset is used as the base frame number. Then, sensitivity tests for overheating area identification are performed near this base frame number (e.g., the base frame number plus or minus 3 frames). For several video segments in the historical fault samples where local overheating is known to exist, each candidate frame number is used for judgment, and the correct recognition rate and false alarm rate at each frame number are calculated. The number of frames with the highest correct recognition rate and a false alarm rate lower than a preset tolerance (e.g., a false alarm rate not exceeding 5%) is selected as the preset number of consecutive frames. If multiple frames meet the condition, the minimum value is taken to improve detection sensitivity. The true value of the overheated area in the historical fault samples is taken as a reference standard using data synchronously collected by the infrared thermal imager. When the temperature of the corresponding area in the infrared image exceeds the average temperature of the equipment during normal operation by more than 20 degrees Celsius, it is determined to be true overheating.

[0118] Traverse all consecutive frame groups in the video signal segment (each group contains a preset number of consecutive frames). If the grayscale value of each frame in a certain group is greater than the grayscale threshold, it is determined that the area corresponding to that group has an overheating phenomenon, and an overheating area identifier is generated to represent the true overheating area; if no consecutive frame group meets the condition, an overheating area identifier is generated to represent the false overheating area.

[0119] Based on the characteristics of minor vibrations and the identification of overheated areas, fault classification is performed using pre-constructed fault mapping rules. The fault mapping rule is a set of decision rules; the inputs are the characteristics of minor vibrations and the identification of overheated areas, and the output is the type of minor vibration fault. The process involves matching the input feature values ​​against the conditions in the rules one by one; if a match is found, the corresponding fault type label is output.

[0120] The fault mapping rule construction process is as follows: Historical vibration features, historical overheating identifiers, and historical fault types are obtained from no fewer than 200 sets of historical fault event samples. The fault types are labeled with true values ​​from on-site maintenance records or infrared thermal imaging verification results. After obtaining the historical vibration features, missing values ​​are detected for each feature. If the percentage of missing values ​​is less than 5%, they are filled with the feature mean; otherwise, samples containing missing values ​​are deleted. Since the decision tree algorithm is not affected by the feature dimensions, normalization is not required. Historical overheating identifiers are encoded with integers (1 for true, 0 for false). Historical vibration features are extracted from historical waveform signal segments through the same frequency domain transformation. Historical overheating identifiers are determined using the same grayscale threshold. Historical fault types are obtained through manual annotation or equipment operation records. Using historical vibration features and historical overheating identifiers as input and historical fault types as output, the initial mapping rules are generated using the CART decision tree algorithm. The CART decision tree algorithm starts from the root node, traverses the possible values ​​of each feature, selects the feature that reduces the impurity (Gini coefficient) of the child node the most as the splitting feature, and recursively generates a binary tree until the samples in each leaf node belong to the same category or a preset depth is reached, resulting in an initial rule set. Based on the initial rule set, redundant branches are removed using a cost-complexity pruning algorithm. This algorithm calculates the weighted sum of the prediction error of each subtree on the training data and the subtree's complexity, selecting the subtree with the smallest weighted sum as the pruned tree, thus obtaining the fault mapping rule. This rule is used to map minor vibration features and overheating indicators to specific micro-vibration fault types, achieving refined fault classification.

[0121] It should be noted that all parameters in this invention that use the 95th percentile as a threshold, including the fitting residual threshold, inter-frame distance threshold, time deviation threshold, correlation strength threshold, and grayscale threshold, are selected based on the distribution characteristics of the statistics under normal operating conditions. This quantile corresponds to the upper limit of the normal fluctuation range, and exceeding this value is considered abnormal.

[0122] Reference Figure 2 The second embodiment of the present invention provides a power equipment fault labeling system based on waveform and UAV video fusion, comprising:

[0123] The data acquisition module is used to acquire waveform amplitude sequences and video image sequences;

[0124] The residual analysis module is used to preprocess the waveform amplitude sequence to obtain abnormal fluctuation records and align the timestamps with the video image sequence to obtain a preliminary fluctuation sequence.

[0125] The period calculation module is used to extract abnormal features from the preliminary fluctuation sequence to obtain a fluctuation feature sequence, and to calculate the main period of the fluctuation feature sequence to obtain a periodic pattern distribution.

[0126] An anomaly filtering module is used to perform anomaly frame filtering by analyzing the pixel displacement and color difference changes of adjacent frames in the video image sequence to obtain anomaly image feature set, and to perform timestamp matching based on the anomaly image feature set and the periodic pattern distribution to obtain anomaly association mapping;

[0127] The dimension reduction fusion module is used to perform feature fusion based on the abnormal correlation mapping to obtain a dimension reduction fusion matrix and filter out abnormal correlation regions using a preset correlation strength threshold.

[0128] The preliminary classification module is used to perform clustering on the abnormal associated regions to obtain initial abnormal clusters, analyze the abnormal duration, fluctuation changes and image changes of the initial abnormal clusters to obtain cluster state vectors, and query a preset preliminary fault type library based on the cluster state vectors to obtain abnormal classification results;

[0129] The output module is used to subdivide the fault types based on the anomaly classification results to obtain the micro-vibration fault types.

[0130] It should be noted that the power equipment fault labeling system based on waveform and UAV video fusion provided in this embodiment of the invention is used to execute all the process steps of the power equipment fault labeling method based on waveform and UAV video fusion in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0131] It should be noted that the system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0132] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for labeling faults in power equipment based on waveform and UAV video fusion, characterized in that, include: Obtain waveform amplitude sequences and video image sequences; The waveform amplitude sequence is preprocessed to obtain abnormal fluctuation records, and the timestamps are aligned with the video image sequence to obtain a preliminary fluctuation sequence; Anomaly features are extracted from the initial fluctuation sequence to obtain a fluctuation feature sequence, and the principal period of the fluctuation feature sequence is calculated to obtain a periodic pattern distribution; An abnormal image feature set is obtained by analyzing the pixel displacement and color difference changes of adjacent frames in the video image sequence to perform abnormal frame screening. An abnormal association mapping is then obtained by timestamp matching based on the abnormal image feature set and the periodic pattern distribution. This includes: calculating the pixel displacement vectors of adjacent frames in the video image sequence using the Lucas-Kanade optical flow method to obtain a pixel displacement sequence; calculating the grayscale difference between adjacent frames in the video image sequence to obtain a color difference change sequence; combining the pixel displacement sequence and the color difference change sequence in chronological order to obtain a frame feature vector sequence; calculating the Euclidean distance between the feature vectors of adjacent frames in the frame feature vector sequence to obtain an inter-frame distance sequence; and selecting video frames in the inter-frame distance sequence whose inter-frame distance is greater than a preset inter-frame distance threshold to obtain abnormal image features. The system collects a large number of video sequences during the equipment's fault-free operation, calculates the inter-frame distance threshold for each historical sequence using the same process, and takes the 95th percentile of all historical inter-frame distance values ​​as the inter-frame distance threshold. Based on the abnormal image feature set and the periodic pattern distribution, a dynamic time warping algorithm is used to perform timestamp matching to obtain an abnormal association mapping. The dynamic time warping algorithm uses the timestamp of each abnormal frame in the abnormal image feature set as the first time series and the timestamp corresponding to each periodic peak in the periodic pattern distribution as the second time series to construct a distance matrix, calculate the cumulative distance matrix, and find the path with the minimum cumulative distance from the upper left corner to the lower right corner, thereby achieving non-linear alignment of the first time series and the second time series on the time axis. Based on the abnormal correlation mapping, feature fusion is performed to obtain a dimension-reduced fusion matrix, and abnormal correlation regions are filtered out using a preset correlation strength threshold. Clustering is performed on the abnormal associated regions to obtain initial abnormal clusters. The abnormal duration, fluctuation changes and image changes of the initial abnormal clusters are analyzed to obtain cluster state vectors. Based on the cluster state vectors, a preset preliminary fault type library is queried to obtain the abnormal classification results. Based on the anomaly classification results, the fault types are further subdivided to obtain micro-vibration fault types.

2. The method for power equipment fault labeling based on waveform and UAV video fusion as described in claim 1, characterized in that, The preprocessing of the waveform amplitude sequence to obtain abnormal fluctuation records and aligning their timestamps with the video image sequence to obtain a preliminary fluctuation sequence includes: Based on the waveform amplitude sequence, noise is filtered using a Gaussian filtering algorithm to obtain a clean waveform sequence. The pure waveform sequence is time-series segmented according to a preset sliding time window, and the waveform peaks in each window are extracted to obtain a set of waveform peaks; Based on the waveform peak set, a spectrum transformation is performed using the Fast Fourier Transform algorithm to obtain a spectrum feature sequence. Then, based on the spectrum feature sequence, a linear fitting is performed using the least squares method to obtain a linear fitting sequence. The difference between the spectral feature sequence and the linear fitting sequence is calculated point by point to obtain the fitting residual sequence. Then, time nodes corresponding to fitting residuals greater than a preset fitting residual threshold are selected to identify abnormal time nodes. The consecutive abnormal time nodes are merged into intervals to obtain abnormal fluctuation records. The abnormal fluctuation records and the video image sequence are then timestamped to obtain a preliminary fluctuation sequence.

3. The method for power equipment fault labeling based on waveform and UAV video fusion according to claim 2, characterized in that, The step of extracting abnormal features from the initial fluctuation sequence to obtain a fluctuation feature sequence, and calculating the principal period of the fluctuation feature sequence to obtain a periodic pattern distribution, includes: Calculate the mean and standard deviation of the amplitude of the preliminary fluctuation sequence within the sliding time window, and extract the extreme points of fluctuations whose amplitude exceeds the mean plus twice the standard deviation to obtain the set of abnormal extreme points; Based on the set of abnormal extreme points, envelope feature vectors are extracted using the Hilbert transform algorithm to obtain the envelope feature vector set. Calculate the Euclidean distance between each sliding time window in the envelope feature vector set to obtain the sliding distance matrix; The slip distance matrix is ​​clustered using a hierarchical clustering algorithm to obtain the fluctuation characteristic sequence; Based on the fluctuation characteristic sequence, the principal period is calculated using a pre-constructed autoregressive model to obtain the periodic pattern distribution.

4. The method for power equipment fault labeling based on waveform and UAV video fusion according to claim 3, characterized in that, The process of constructing the autoregressive model includes: Historical fluctuation sequences are obtained, the model order is selected using the Akaike Information Criterion, and the coefficients are estimated using the least squares method based on the historical fluctuation sequences and the model order to obtain the autoregressive model coefficients. Based on the historical fluctuation sequence, the model order, and the autoregressive model coefficients, the model fit value is calculated, and the difference between the model fit value and the historical fluctuation sequence is calculated to obtain the model residual sequence. Based on the model residual sequence, the model validity was verified using the Ljung-Box test method to obtain the white noise test value; When the white noise test value is greater than the preset white noise test threshold, the model is deemed valid, and the constructed autoregressive model is obtained. When the white noise test value is not greater than the white noise test threshold, the order selection and coefficient estimation are repeated until the white noise test value is greater than the white noise test threshold, thus obtaining the autoregressive model.

5. The method for power equipment fault labeling based on waveform and UAV video fusion according to claim 1, characterized in that, The step of performing feature fusion based on the abnormal correlation mapping to obtain a dimensionality-reduced fusion matrix and filtering out abnormal correlation regions using a preset correlation strength threshold includes: Calculate the time difference between each pair of matching points in the abnormal association mapping to obtain a time deviation sequence, and filter out the matching point pairs in the time deviation sequence whose time deviation is less than a preset time deviation threshold to obtain valid matching point pairs; The fluctuation features corresponding to the effective matching point pairs are extracted from the periodic pattern distribution to obtain the interval fluctuation feature set, and the abnormal image features corresponding to the effective matching point pairs are extracted from the abnormal image feature set to obtain the interval image feature set; Based on the interval fluctuation feature set and the interval image feature set, a dimension reduction and fusion matrix is ​​obtained by performing dimensionality reduction and fusion using the principal component analysis algorithm; Calculate the Pearson correlation coefficient between the fluctuation features and image features at each position in the dimensionality reduction fusion matrix to obtain the correlation intensity distribution; The regions corresponding to the correlation intensity greater than the preset correlation intensity threshold in the correlation intensity distribution are selected to obtain the abnormal correlation regions.

6. The method for power equipment fault labeling based on waveform and UAV video fusion according to claim 1, characterized in that, The process involves clustering the abnormally associated regions to obtain initial abnormal clusters, analyzing the duration, fluctuations, and image changes of these clusters to obtain cluster state vectors, and querying a pre-defined preliminary fault type library based on these cluster state vectors to obtain anomaly classification results, including: The fluctuation characteristics and image features within the abnormal correlation region are clustered using the K-means clustering algorithm to obtain initial abnormal clusters; Calculate the mean interval duration of each cluster within the initial abnormal cluster to obtain the average cluster duration; By using the least squares method, linear fitting is performed on the fluctuation characteristics and image characteristics of all intervals in each cluster within the initial anomaly cluster to obtain the fluctuation fitting line and the image fitting line. The slopes of the fluctuation fitting line and the image fitting line are then extracted to obtain the fluctuation change slope and the image change slope. The average duration of the cluster, the slope of the fluctuation change, and the slope of the image change are combined into a cluster state vector; Based on the cluster state vector, fault mode matching is performed using a preset preliminary fault type library to obtain anomaly classification results.

7. The method for power equipment fault labeling based on waveform and UAV video fusion according to claim 1, characterized in that, The step of subdividing the fault types based on the anomaly classification results to obtain micro-vibration fault types includes: The signal segments corresponding to the anomaly classification results are extracted from the waveform amplitude sequence and the video image sequence, respectively, to obtain waveform signal segments and video signal segments; Based on the waveform signal segment, frequency domain transformation is performed using the Fast Fourier Transform algorithm to obtain the micro-vibration characteristics; When there are consecutive preset number of grayscale values ​​in the video signal segment that are all greater than a preset grayscale threshold, an overheated area identifier is generated to represent the true overheated area. When there are no consecutive frames in the video signal segment whose grayscale values ​​are all greater than the grayscale threshold, a false overheating area identifier is generated. Based on the micro-vibration characteristics and the overheated area identification, the micro-vibration fault type is obtained by classifying the faults using pre-built fault mapping rules.

8. The method for power equipment fault labeling based on waveform and UAV video fusion according to claim 7, characterized in that, The process of constructing the fault mapping rules includes: Acquire historical vibration characteristics, historical overheating indicators, and historical fault types; Using the historical vibration characteristics and historical overheating identifiers as inputs and the historical fault type as outputs, the initial mapping rules are generated through the CART decision tree algorithm to obtain the initial rule set. Based on the initial rule set, redundant branches are removed using a cost-complexity pruning algorithm to obtain fault mapping rules.

9. A power equipment fault labeling system based on waveform and UAV video fusion, characterized in that, include: The data acquisition module is used to acquire waveform amplitude sequences and video image sequences; The residual analysis module is used to preprocess the waveform amplitude sequence to obtain abnormal fluctuation records and align the timestamps with the video image sequence to obtain a preliminary fluctuation sequence. The period calculation module is used to extract abnormal features from the preliminary fluctuation sequence to obtain a fluctuation feature sequence, and to calculate the main period of the fluctuation feature sequence to obtain a periodic pattern distribution. An anomaly filtering module is used to perform anomaly frame filtering by analyzing the pixel displacement changes and color difference changes of adjacent frames in the video image sequence to obtain an anomaly image feature set, and to obtain an anomaly association mapping by performing timestamp matching based on the anomaly image feature set and the periodic pattern distribution. This includes: calculating the pixel displacement vectors of adjacent frames in the video image sequence using the Lucas-Kanade optical flow method to obtain a pixel displacement sequence; calculating the grayscale difference between adjacent frames in the video image sequence to obtain a color difference change sequence; combining the pixel displacement sequence and the color difference change sequence in chronological order to obtain a frame feature vector sequence; calculating the Euclidean distance between the feature vectors of adjacent frames in the frame feature vector sequence to obtain an inter-frame distance sequence; and filtering out video frames in the inter-frame distance sequence whose inter-frame distance is greater than a preset inter-frame distance threshold to obtain anomalies. The image feature set, wherein the inter-frame distance threshold is determined by statistical analysis of video data under historical normal operating conditions, a large number of video sequences during the fault-free operation of the equipment are collected, and the inter-frame distance of each historical sequence is calculated according to the same process. The 95th percentile of all historical inter-frame distance values ​​is taken as the inter-frame distance threshold. Based on the abnormal image feature set and the periodic pattern distribution, timestamp matching is performed through a dynamic time warping algorithm to obtain an abnormal association mapping. The dynamic time warping algorithm uses the timestamp of each abnormal frame in the abnormal image feature set as the first time series and the timestamp corresponding to each periodic peak in the periodic pattern distribution as the second time series to construct a distance matrix, calculate the cumulative distance matrix, and find the path with the minimum cumulative distance from the upper left corner to the lower right corner, thereby achieving non-linear alignment of the first time series and the second time series on the time axis. The dimension reduction fusion module is used to perform feature fusion based on the abnormal correlation mapping to obtain a dimension reduction fusion matrix and filter out abnormal correlation regions using a preset correlation strength threshold. The preliminary classification module is used to perform clustering on the abnormal associated regions to obtain initial abnormal clusters, analyze the abnormal duration, fluctuation changes and image changes of the initial abnormal clusters to obtain cluster state vectors, and query a preset preliminary fault type library based on the cluster state vectors to obtain abnormal classification results; The output module is used to subdivide the fault types based on the anomaly classification results to obtain the micro-vibration fault types.

Images