Fault Association Annotation Method Integrating PMU Waveforms and UAV Video Defects

Abstract

This invention belongs to the field of power system automation technology and relates to a fault association annotation method that integrates PMU waveforms and UAV video defects. The method involves acquiring waveform data collected by the PMU device, detecting abnormal fluctuation segments and marking them as candidate fault evidence collection periods; generating UAV mission instructions based on the changes in the abnormal waveforms to control the UAV to collect video streams from the target equipment; extracting time-domain and frequency-domain feature values ​​from the waveform data, extracting defect feature values ​​of equipment components and inputting them into a pre-built feature matcher, outputting the correspondence and similarity between waveforms and defect features; determining the faulty equipment identifier, fault type code, and fault level value, and encapsulating them into an annotation data package. This invention achieves accurate association annotation between abnormal electrical waveforms and equipment appearance defects, avoiding the discrimination bias of a single data source, improving the accuracy of fault location and the consistency of annotation results, effectively reducing the workload of manual review, and is suitable for intelligent power grid inspection and fault handling scenarios.

Description

Technical Field

[0001] This invention belongs to the field of power system automation technology and relates to a fault association annotation method that integrates PMU waveforms and UAV video defects. Background Technology

[0002] In the field of power system equipment inspection, synchronous phasor measurement units (PMUs) can continuously acquire voltage and current waveform data during power grid operation at a high sampling rate, demonstrating rapid response to instantaneous changes in electrical quantities and effectively capturing abnormal electrical characteristics such as voltage dips, swells, and harmonic distortion. Meanwhile, drone inspection technology, with its advantages of mobility, wide coverage, and rich visual information, can perform close-up photography of the external condition of power equipment, identifying visible defects such as insulator damage, broken conductor strands, and metal fitting corrosion. Currently, PMU devices are mainly deployed in substations or critical nodes, focusing on real-time monitoring of electrical quantities and disturbance alarms, while drone inspection serves as a periodic or emergency means of equipment visual inspection; the two operate independently within the power operation and maintenance system.

[0003] However, existing fault diagnosis methods have significant limitations due to the lack of an effective correlation mechanism between PMU waveform data and UAV video data. While relying solely on PMU waveform analysis can identify electrical anomalies in the power grid, it is difficult to pinpoint the specific equipment components causing the anomaly, nor can it reveal whether there are accompanying defects in the equipment's appearance. Relying solely on UAV video inspections is susceptible to significant image quality fluctuations due to environmental factors such as lighting, weather, and shooting angle, and the lack of electrical features as supporting criteria easily leads to misjudgments or missed detections of defects. Furthermore, video annotation relies heavily on manual experience, and inconsistent annotation standards make it difficult to quickly and accurately incorporate fault information into the operation and maintenance decision-making process.

[0004] Therefore, effectively integrating the electrical fault characteristics inherent in PMU waveforms with the equipment appearance defect characteristics presented by UAV videos, enabling electrical anomalies to guide accurate video inspections, and simultaneously using waveform features to verify and supplement video interpretation results to achieve unified labeling of faulty equipment, fault type, and severity level, is a pressing technical problem in the field of intelligent operation and maintenance of power equipment. Existing methods mostly focus on processing a single data source or simply superimposing two data sources, failing to address the spatiotemporal alignment and semantic matching issues between waveform features and image features, thus limiting the automation level and response speed of fault detection. Summary of the Invention

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] A fault association annotation method that integrates PMU waveforms and UAV video defects includes:

[0007] Step 1: Acquire waveform data collected by the PMU device during the operation of the power equipment, detect abnormal fluctuation segments in the waveform data, and mark the time period corresponding to the abnormal fluctuation segment as the fault candidate evidence collection time period;

[0008] Step 2: Based on the fault candidate evidence collection period and the waveform change pattern of the waveform data in the abnormal fluctuation segment, generate a UAV mission command containing the target device coordinates and gimbal shooting angle, and send the UAV mission command to the UAV control system.

[0009] Step 3: Receive the video stream collected by the drone at the coordinates of the target device, extract the waveform time-domain feature value and waveform frequency-domain feature value in the abnormal fluctuation segment, extract key frame images from the video stream, and perform device component area detection on the key frame images to obtain defect feature values;

[0010] Step 4: Input the waveform time-domain feature value, waveform frequency-domain feature value and defect feature value into the feature matcher, and output the matching result according to the pre-established correspondence rules between waveform features and defect features, including the correspondence and similarity between waveform feature values ​​and defect feature values;

[0011] Step 5: Determine the faulty device identifier, fault type code, and fault level value based on the matching results. The faulty device identifier corresponds to the device component with the highest similarity in the matching results.

[0012] Step 6: Encapsulate the faulty equipment identifier, fault type code, fault level value, fault candidate evidence collection period, waveform time domain feature value, waveform frequency domain feature value, defect feature value, and key frame image into a labeling data package, and store the labeling data package in the fault record database.

[0013] Further, step 2 includes:

[0014] The waveform abrupt change type is identified based on the waveform change pattern within the abnormal fluctuation range of the waveform data. The waveform abrupt change types include voltage sag, voltage swell, harmonic distortion, and pulse spike.

[0015] For each type of waveform abrupt change, a corresponding device coordinate offset and gimbal angle offset are preset;

[0016] The original equipment installation coordinates of the abnormal waveform are determined based on the fault candidate evidence collection period, and the target equipment coordinates are obtained by superimposing the original equipment installation coordinates with the equipment coordinate offset.

[0017] The gimbal shooting angle is obtained by adding the gimbal angle offset to the original default angle of the gimbal.

[0018] The target device coordinates and gimbal shooting angle are encoded into UAV mission commands.

[0019] Furthermore, step 3 involves extracting the waveform time-domain and frequency-domain feature values ​​within the abnormal fluctuation segment, including:

[0020] Waveform amplitude is sampled sequentially along the time axis of the abnormal fluctuation segment. Local maxima and local minima in the waveform amplitude sequence are recorded. The maximum value in the difference sequence between local maxima and local minima is taken as the fluctuation depth in the waveform time domain feature value. The shortest time required for the waveform amplitude to rise from a local minima to a local maxima is recorded as the rise rate feature in the waveform time domain feature value. The shortest time required for the waveform amplitude to fall from a local maxima to a local minima is recorded as the fall rate feature in the waveform time domain feature value.

[0021] The waveform amplitude sequence within the abnormal fluctuation segment is decomposed according to a preset frequency window, and the fundamental energy ratio, second harmonic energy ratio, third harmonic energy ratio, and total harmonic energy distortion coefficient are extracted as waveform frequency domain feature values.

[0022] Furthermore, step 4 includes:

[0023] Step 41: Obtain the pre-built fault feature mapping library, which contains multiple mapping records, consisting of waveform feature templates and defect feature templates, which respectively contain standard waveform time domain parameter ranges, standard waveform frequency domain parameter ranges, and standard defect image parameter ranges;

[0024] Step 42: Compare the extracted waveform time-domain feature values ​​with the standard waveform time-domain parameter range in each mapping record. If the waveform time-domain feature value falls within the standard waveform time-domain parameter range, a time-domain hit mark is generated; otherwise, a time-domain miss mark is generated.

[0025] Step 43: Compare the extracted waveform frequency domain feature values ​​with the standard waveform frequency domain parameter range in the same mapping record. If the waveform frequency domain feature value falls within the standard waveform frequency domain parameter range, a frequency domain hit mark is generated; otherwise, a frequency domain miss mark is generated.

[0026] Step 44: Compare the extracted defect feature value with the standard defect image parameter range in the same mapping record. If the defect feature value falls within the standard defect image parameter range, a defect hit mark is generated; otherwise, a defect miss mark is generated.

[0027] Step 45: For each mapping record, count the total number of hits of time-domain hit markers, frequency-domain hit markers, and defect hit markers, and determine the mapping record with the highest total number of hits as the candidate matching record;

[0028] Step 46: When the candidate matching record is unique, output the correspondence between the waveform feature template and the defect feature template in the candidate matching record as the matching result, and use the ratio of the total number of hits to the total number of mapped records as the similarity. When the candidate matching record is not unique, compare the interval width of the waveform feature template and the defect feature template in each candidate matching record, select the candidate matching record with the smallest interval width as the matching result, output the correspondence in the matching result, and use the ratio of the total number of hits in the matching result to the total number of mapped records as the similarity.

[0029] Furthermore, step 5 includes:

[0030] Extract the most similar correspondences from the matching results, including the association pairs between waveform feature values ​​and defect feature values;

[0031] The predefined equipment component mapping table is searched based on the defect feature value in the association pair. The equipment component mapping table records the corresponding entries between the defect feature value and the equipment component name. The faulty equipment identifier is read from the corresponding entry.

[0032] Based on the defect feature values ​​in the associated pair, look up the predefined defect type mapping table and read the fault type code from the defect type mapping table;

[0033] The predefined grade mapping table is searched based on the waveform time-domain feature value and waveform frequency-domain feature value in the association pair. The grade mapping table records the grade values ​​corresponding to different waveform time-domain feature value intervals and waveform frequency-domain feature value intervals. The grade value corresponding to the interval where the waveform time-domain feature value and waveform frequency-domain feature value are located is determined as the fault grade value.

[0034] Furthermore, the specific method for encapsulating the labeled data packet in step 6 is as follows:

[0035] Create a data container according to the predetermined data exchange format, set the header segment and payload segment in the data container, write the fault candidate evidence collection period into the timestamp field of the header segment, and write the fault device identifier, fault type code and fault level value into the identifier field of the header segment.

[0036] In the load segment, waveform feature blocks and image feature blocks are established. Waveform time-domain feature values ​​and waveform frequency-domain feature values ​​are written into the waveform feature blocks, and defect feature values ​​and key frame images are written into the image feature blocks. The header segment and load segment are combined to form a label data packet, and the label data packet is appended to the end record position of the fault record database.

[0037] Furthermore, the specific method for detecting abnormal fluctuation segments in the waveform data in step 1 is as follows:

[0038] Traverse each sampling point in the waveform data, calculate the amplitude difference between the current sampling point and the previous sampling point, and mark the current sampling point as an abnormal starting point when the absolute value of the amplitude difference exceeds the preset amplitude change threshold;

[0039] Starting from the abnormal starting point, continue to traverse subsequent sampling points and calculate the average amplitude change rate between each subsequent sampling point and the abnormal starting point. When the average amplitude change rate is continuously less than the preset recovery threshold and the duration exceeds the preset stabilization duration, mark the last sampling point that meets the recovery threshold as the abnormal ending point; extract the waveform segment between the abnormal starting point and the abnormal ending point as the abnormal fluctuation segment.

[0040] Furthermore, step 3, which involves extracting keyframe images from the video stream and performing device component region detection on the keyframe images to obtain defect feature values, includes:

[0041] Candidate frame images are extracted from the video stream at fixed time intervals. The pixel difference between adjacent candidate frame images is calculated. Candidate frame images whose pixel difference exceeds a preset difference threshold are identified as key frame images. Pixel gradient operations are performed on the key frame images to obtain edge feature maps. The edge feature maps are overlapped and matched with pre-stored device component templates to determine the region where the device component is located.

[0042] Sub-images of the area where the device component is located are cropped from the keyframe image. Texture analysis is performed on the sub-images to extract texture feature parameters. Morphological analysis is performed on the sub-images to extract geometric feature parameters. The texture feature parameters and geometric feature parameters are combined into defect feature values.

[0043] Furthermore, the mapping records in the fault feature mapping library are pre-established in the following way:

[0044] Collect historical failure cases, each of which includes historical waveform clips of the PMU and historical video clips of the drone;

[0045] Extract waveform time-domain and frequency-domain historical feature values ​​from PMU historical waveform segments, and extract defect historical feature values ​​from UAV historical video segments;

[0046] The waveform time-domain historical feature values ​​and waveform frequency-domain historical feature values ​​are combined into a waveform feature template, and the defect historical feature values ​​are combined into a defect feature template.

[0047] Based on the fault type labeling of historical fault cases, the waveform feature template and defect feature template corresponding to the same fault type are associated and stored as a mapping record;

[0048] For each mapping record, the minimum and maximum values ​​of the waveform time-domain historical feature values ​​in the waveform feature template are taken to form the standard waveform time-domain parameter interval. The minimum and maximum values ​​of the waveform frequency-domain historical feature values ​​are taken to form the standard waveform frequency-domain parameter interval. The minimum and maximum values ​​of the defect historical feature values ​​are taken to form the standard defect image parameter interval.

[0049] This invention belongs to the field of power system automation technology and relates to a fault association annotation method that integrates PMU waveforms and UAV video defects. The method involves acquiring waveform data collected by the PMU device, detecting abnormal fluctuation segments and marking them as candidate fault evidence collection periods; generating UAV mission instructions based on the changes in the abnormal waveforms to control the UAV to collect video streams from the target equipment; extracting time-domain and frequency-domain feature values ​​from the waveform data, extracting defect feature values ​​of equipment components, inputting them into a pre-built feature matcher, and outputting the correspondence and similarity between the waveform and defect features; determining the faulty equipment identifier, fault type code, and fault level value, and encapsulating them into an annotation data package. This invention achieves accurate association annotation between abnormal electrical waveforms and equipment appearance defects, avoiding the discrimination bias of a single data source, improving the accuracy of fault location and the consistency of annotation results, effectively reducing the workload of manual review, and is suitable for intelligent power grid inspection and fault handling scenarios. Attached Figure Description

[0050] Figure 1 A flowchart illustrating the fault association annotation method for fusing PMU waveforms and UAV video defects, as claimed in an embodiment of the present invention.

[0051] Figure 2 The second flowchart is shown for a fault association annotation method that integrates PMU waveforms and UAV video defects, as claimed in an embodiment of the present invention.

[0052] Figure 3 The third flowchart is a fault association annotation method for fusing PMU waveforms and UAV video defects, as claimed in an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0054] The terms "first," "second," and "third" in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of those features. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications in the embodiments of this application, such as up, down, left, right, front, back, etc., are only used to explain the relative positional relationships and movements between components in a specific orientation as shown in the accompanying drawings. If the specific orientation changes, the directional indications will change accordingly. Furthermore, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0055] References to embodiments herein mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0056] According to the first embodiment of the present invention, referring to Figure 1 This invention claims protection for a fault association annotation method that integrates PMU waveforms and UAV video defects, comprising:

[0057] Step 1: Acquire waveform data collected by the PMU device during the operation of the power equipment, detect abnormal fluctuation segments in the waveform data, and mark the time period corresponding to the abnormal fluctuation segment as the fault candidate evidence collection time period;

[0058] Step 2: Based on the fault candidate evidence collection period and the waveform change pattern of the waveform data in the abnormal fluctuation segment, generate a UAV mission command containing the target device coordinates and gimbal shooting angle, and send the UAV mission command to the UAV control system.

[0059] Step 3: Receive the video stream collected by the drone at the coordinates of the target device, extract the waveform time-domain feature value and waveform frequency-domain feature value in the abnormal fluctuation segment, extract key frame images from the video stream, and perform device component area detection on the key frame images to obtain defect feature values;

[0060] Step 4: Input the waveform time-domain feature value, waveform frequency-domain feature value and defect feature value into the feature matcher, and output the matching result according to the pre-established correspondence rules between waveform features and defect features, including the correspondence and similarity between waveform feature values ​​and defect feature values;

[0061] Step 5: Determine the faulty device identifier, fault type code, and fault level value based on the matching results. The faulty device identifier corresponds to the device component with the highest similarity in the matching results.

[0062] Step 6: Encapsulate the faulty equipment identifier, fault type code, fault level value, fault candidate evidence collection period, waveform time domain feature value, waveform frequency domain feature value, defect feature value, and key frame image into a labeling data package, and store the labeling data package in the fault record database.

[0063] In this embodiment, a PMU device is installed on a tower of a 220 kV transmission line. The PMU device continuously collects three-phase voltage and current waveform data at a sampling rate of 4,000 points per second. The PMU device transmits the real-time waveform data to a regional data aggregation server via optical fiber.

[0064] In step 1, the detection process on the data collection server reads the latest received waveform data segment every power frequency cycle. A sliding window analyzer runs within this process, with a window width corresponding to three power frequency cycles. The analyzer calculates the instantaneous amplitude of each sampling point within the current window and compares the amplitude of each sampling point with the amplitude of the same phase point in the previous cycle. When the amplitude change of a sampling point exceeds a pre-set action threshold, the analyzer records the timestamp of that sampling point as the abnormal start time. Starting from the abnormal start time, the analyzer continues to traverse the sampling points and continuously calculates the amplitude difference between each subsequent sampling point and the sampling point at the abnormal start time. When the amplitude difference of one hundred consecutive sampling points is less than the recovery threshold, the analyzer uses the timestamp of the last of these one hundred sampling points as the abnormal end time. The waveform segment formed by all sampling points between the abnormal start time and the abnormal end time is extracted and stored in a temporary buffer along with the start and end times of the waveform segment. This time period is then marked as a fault candidate evidence collection period.

[0065] In step 2, the processor reads the waveform data of the abnormal fluctuation segment from the temporary buffer and first identifies the overall shape of the waveform segment. The processor calculates the peak-to-valley difference of the waveform within the abnormal fluctuation segment and classifies the waveform shape into a voltage sag type based on the magnitude of the peak-to-valley difference and the steepness of the waveform change. The processor then looks up the corresponding coordinate offset from a pre-stored mapping configuration table based on the voltage sag type. This mapping configuration table records the coordinate offset value along the line direction (two tower spacings) and the angle offset value of the pan / tilt unit tilting downwards by fifteen degrees for the voltage sag type. Simultaneously, the processor checks the equipment ledger... The processor retrieves the original equipment installation location coordinates monitored by the PMU from the database; it adds these original equipment installation location coordinates to the aforementioned coordinate offset value to obtain the target equipment coordinates; it then adds the default gimbal horizontal angle (0 degrees) and pitch angle (0 degrees) of the UAV to the gimbal pitch angle tilted downwards by 15 degrees to obtain the gimbal shooting angle; the processor encodes the target equipment coordinates and the gimbal shooting angle into a task command message according to the UAV control protocol. The message header contains a command type identifier and a priority identifier, and the message body sequentially writes the longitude, latitude, and altitude of the target equipment, as well as the gimbal horizontal angle, pitch angle, and zoom magnification. This task command is sent to the UAV control system via a message queue.

[0066] When step 3 is executed, after receiving the mission instruction, the UAV control system dispatches a UAV in standby mode to take off and fly to the location of the target equipment according to the coordinates in the instruction. After hovering, the UAV adjusts the visible light camera gimbal according to the gimbal shooting angle in the instruction so that the camera lens is aimed at the component of the equipment to be inspected. The drone begins recording a video stream, which is transmitted back to the ground server in real time with timestamps attached to each frame. Upon receiving the video stream, the server performs two simultaneous processing steps: First, it extracts the time-domain and frequency-domain feature values ​​of the waveform within the abnormal fluctuation segment from the previously cached waveform data. Specifically, it extracts the amplitude values ​​of all sampling points within the abnormal fluctuation segment, identifies the maximum and minimum values ​​in the amplitude sequence, and calculates the difference between the maximum and minimum values ​​as the fluctuation depth. It then finds the minimum number of sampling points the waveform passes through to rise from the minimum to the maximum value, divides this number by the sampling rate to obtain the rise time, and uses the reciprocal of the rise time as the rise rate feature. Finally, it finds the minimum number of sampling points the waveform passes through to fall from the maximum to the minimum value, calculates the reciprocal of the fall rate as the fall rate feature, and performs a Fast Fourier Transform on the amplitude sequence within the abnormal fluctuation segment to obtain the amplitudes of the fundamental frequency component, the second harmonic component, and the third harmonic component, and calculates their respective amplitudes. The ratio of each component amplitude to the total amplitude is calculated as the fundamental energy proportion, second harmonic energy proportion, and third harmonic energy proportion, and the sum of the harmonic energy proportions is used as the total harmonic energy distortion coefficient; the second path extracts keyframe images from the video stream: every fifty frames, a candidate image is extracted, and the sum of the absolute values ​​of the pixel grayscale differences between adjacent candidate images is calculated. When the sum exceeds a set threshold, the next candidate image is determined as the keyframe image; the gradient amplitude is calculated using the Sobel operator on the keyframe image to obtain an edge map, and the edge map is matched with a pre-stored insulator string template using a sliding window correlation. The window position with the highest matching degree is the region where the insulator string is located; the region is cropped from the original keyframe image to obtain a sub-image, and the variance of the pixel grayscale values ​​in the sub-image is calculated as the texture feature parameter. The aspect ratio of the bounding rectangle of the connected components in the sub-image is calculated as the geometric feature parameter, and the texture feature parameter and the geometric feature parameter are combined into a defect feature value vector.

[0067] In step 4, the seven values ​​obtained above—wave depth, rise rate feature, fall rate feature, fundamental energy proportion, second harmonic energy proportion, third harmonic energy proportion, and total harmonic distortion coefficient—are combined to form a waveform feature vector. Texture feature parameters and geometric feature parameters are combined to form a defect feature vector. The processor calls a feature matcher, which internally loads a pre-built fault feature mapping library. Each record in the library contains a waveform feature template and a corresponding defect feature template. The matcher sequentially compares each component in the waveform feature vector with the interval range of the waveform feature template in each record, and simultaneously compares the defect feature vector with the interval range of the defect feature template. For each record, the number of components in the waveform feature vector falling within the corresponding interval range and the number of components in the defect feature vector falling within the corresponding interval range are calculated, and these two are added together to obtain the hit count. The record with the highest hit count is determined as the matching result, and the correspondence between the waveform feature and the defect feature is read from this record. The hit count is then divided by the total number of records in the mapping library to obtain the similarity ratio.

[0068] In step 5, the processor extracts the corresponding relationship from the matching results. This relationship indicates that the current waveform features are associated with the surface damage defect features of the insulator. Based on the defect feature value in this relationship, the processor searches in the equipment component mapping table. The equipment component mapping table is stored in key-value pairs, where each record's key is the defect feature code and the value is the string of the equipment component name. The query results in the faulty equipment identifier being the B-phase insulator string of the line tower. The processor then searches in the defect type mapping table based on the texture parameter range code in the defect feature value, obtaining the fault type code as insulator damage. The processor compares the fluctuation depth value in the waveform time domain feature value and the total harmonic energy distortion coefficient in the waveform frequency domain feature value with the conditions in the level mapping table. The level mapping table pre-sets sub-levels corresponding to different fluctuation depth ranges and sub-levels corresponding to different total harmonic energy distortion coefficient ranges. The higher sub-level is taken as the fault level value, for example, the output level value is level three.

[0069] In step 6, the processor creates a data container object that follows a custom binary exchange format. The data container is divided into a header and a payload. The header contains the start and end times of the fault candidate evidence period, as well as the fault equipment identifier (line tower B-phase insulator string), fault type code (INS_BRK_03), and fault level value (level 3). The payload is divided into two blocks: the waveform feature block contains, in sequence, wave depth, rise rate characteristics, fall rate characteristics, fundamental energy percentage, second harmonic energy percentage, third harmonic energy percentage, and total harmonic distortion coefficient; the image feature block first contains the texture and geometric feature parameters from the defect feature values, and then contains the compressed JPEG data block of the keyframe image. The header and payload are combined into a complete binary data packet, and the database write interface is called to append the data packet to the end table of the fault record database. Each record also includes a system timestamp at the time of writing as an archive marker.

[0070] Further, step 2 includes:

[0071] The waveform abrupt change type is identified based on the waveform change pattern within the abnormal fluctuation range of the waveform data. The waveform abrupt change types include voltage sag, voltage swell, harmonic distortion, and pulse spike.

[0072] For each type of waveform abrupt change, a corresponding device coordinate offset and gimbal angle offset are preset;

[0073] The original equipment installation coordinates of the abnormal waveform are determined based on the fault candidate evidence collection period, and the target equipment coordinates are obtained by superimposing the original equipment installation coordinates with the equipment coordinate offset.

[0074] The gimbal shooting angle is obtained by adding the gimbal angle offset to the original default angle of the gimbal.

[0075] The target device coordinates and gimbal shooting angle are encoded into UAV mission commands.

[0076] In this embodiment, in step 2, after receiving the waveform data of the fault candidate evidence period, the processor first starts the waveform mutation type identification subroutine. This subroutine matches the waveform amplitude sequence within the abnormal fluctuation segment with five preset waveform templates: the voltage sag template is defined as the amplitude dropping to below 50% of the normal value within half a power frequency cycle and recovering after three power frequency cycles; the voltage swell template is defined as the amplitude rising to above 120% of the normal value and lasting for two power frequency cycles; the harmonic distortion template is defined as the waveform exhibiting obvious spur-like high-frequency oscillations with more than six zero-crossing jitters within each half power frequency cycle; the pulse spike template is defined as the amplitude of a single sampling point exceeding five times the normal value and the amplitudes of the two preceding and following sampling points being normal. The subroutine calculates the correlation coefficient between the abnormal fluctuation segment waveform and each template, and selects the type corresponding to the template with the highest correlation coefficient as the waveform mutation type.

[0077] Assuming the identification result is a voltage sag type, the processor reads the preset parameters corresponding to this type from the waveform mutation type configuration table. The configuration table is a key-value database stored on the local disk. The device coordinate offset corresponding to the voltage sag type is set to be offset by three tower spacings along the line direction, in the downstream direction away from the PMU installation point, with an offset of 150 meters due east and 0 meters due north. The pan-tilt unit angle offset is set to zero degrees in the horizontal direction and 20 degrees downward in the pitch direction. Simultaneously, the processor queries the device ledger database for the original device installation coordinates bound to the PMU device during system registration. The original device installation coordinates are longitude 118.723 degrees, latitude 31.981 degrees, and altitude 15 meters. The processor adds the longitude change value corresponding to the offset to the original longitude, adds the latitude change value corresponding to the offset to the original latitude, and keeps the altitude unchanged, to obtain the target device coordinates as longitude 118.738 degrees, latitude 31.981 degrees, and altitude 15 meters. The original default angle of the gimbal is set to zero degrees horizontal (due north) and zero degrees pitch (horizontal forward). The default pitch angle of zero degrees is adjusted downward by 20 degrees by adding the offset, resulting in a gimbal shooting angle of zero degrees horizontal and -20 degrees pitch.

[0078] The processor then generates drone mission instructions. These instructions are in JSON text format, but are converted to binary TLV format for compatibility with older drone systems. The first field of the instruction is the type identifier, with a value of 0x01 indicating a refined inspection task. The second field is the priority, with a value of 0x02 indicating high priority. Subsequently, three double-precision floating-point numbers representing the target device coordinates are written, followed by two single-precision floating-point numbers representing the gimbal's horizontal and pitch angles. The processor also writes the waveform mutation type code VT_DIP into the instruction's extended field. After the complete instruction assembly is complete, it is sent to the drone control system's mission receiving port via a TCP / IP connection.

[0079] Furthermore, step 3 involves extracting the waveform time-domain and frequency-domain feature values ​​within the abnormal fluctuation segment, including:

[0080] Waveform amplitude is sampled sequentially along the time axis of the abnormal fluctuation segment. Local maxima and local minima in the waveform amplitude sequence are recorded. The maximum value in the difference sequence between local maxima and local minima is taken as the fluctuation depth in the waveform time domain feature value. The shortest time required for the waveform amplitude to rise from a local minima to a local maxima is recorded as the rise rate feature in the waveform time domain feature value. The shortest time required for the waveform amplitude to fall from a local maxima to a local minima is recorded as the fall rate feature in the waveform time domain feature value.

[0081] The waveform amplitude sequence within the abnormal fluctuation segment is decomposed according to a preset frequency window, and the fundamental energy ratio, second harmonic energy ratio, third harmonic energy ratio, and total harmonic energy distortion coefficient are extracted as waveform frequency domain feature values.

[0082] In this embodiment, the processor first reads the waveform amplitude sequence of the abnormal fluctuation segment from the buffer. This sequence contains all continuously sampled points from the start time to the end time. For example, if the abnormal fluctuation segment lasts for 0.05 seconds and the sampling rate is 4,000 points per second, then there are a total of 200 sampling points. The processor iterates through the amplitudes of these 200 sampling points to find local maxima and local minima. The method for determining local maxima is as follows: for internal sampling points, if the amplitude of the current point is greater than the previous point and greater than the next point, then the current point is a local maximum; for sequence endpoints, endpoints are not considered extreme points. Similarly, the method for determining local minima is as follows: if the amplitude of the current point is less than the previous point and less than the next point, then it is a local minimum.

[0083] After obtaining all local maxima and local minima, the processor subtracts each local maximum from its next adjacent local minimum, resulting in one set of differences; then, it subtracts each local minimum from its next adjacent local maximum, resulting in another set of differences. All values ​​in these two sets of differences are compared, and the largest value is taken as the fluctuation depth feature value. For example, if the maximum local maximum is 235 amperes, and the adjacent local minimum is 120 amperes, the difference is 115 amperes; if another local maximum is 200 amperes, and the adjacent local minimum is 90 amperes, the difference is 110 amperes; then the fluctuation depth is taken as 115 amperes.

[0084] Next, the processor calculates the rise rate feature. Starting from each local minimum, it iterates through the sampling points until the next local maximum is encountered, recording the number of sampling points required to rise from a minimum to a maximum. The rise time is obtained by dividing this number of sampling points by the sampling rate. If multiple minimum-maximum pairs have multiple rise times, the minimum rise time is taken, and its reciprocal is used as the rise rate feature. For example, if the minimum rise time is 0.002 seconds, the rise rate feature is 500. Similarly, the fall rate feature is calculated by iterating from each local maximum to the next local minimum, recording the number of sampling points required for the fall, calculating the fall time, and taking the minimum fall time as its reciprocal as the fall rate feature.

[0085] For waveform frequency domain feature extraction, the processor groups the amplitude values ​​of two hundred sampling points within the abnormal fluctuation segment into an array. The processor calls a Discrete Fourier Transform program, which outputs the amplitude of each frequency component. The processor extracts the amplitude of the fundamental frequency component (50 Hz), the amplitude of the second harmonic component (100 Hz), the amplitude of the third harmonic component (150 Hz), and the square root of the sum of the squares of the amplitudes of all other frequency components as the effective value of the total harmonic components. The fundamental frequency energy proportion is calculated as: the square of the fundamental frequency component amplitude divided by the sum of the squares of the amplitudes of all frequency components; the second harmonic energy proportion is calculated as: the square of the second harmonic component amplitude divided by the sum of the squares of the amplitudes of all frequency components; the third harmonic energy proportion is calculated similarly. The total harmonic distortion coefficient is calculated as: the square root of the sum of the squares of the amplitudes of all harmonic components excluding the fundamental frequency divided by the fundamental frequency amplitude; the processor stores these calculation results as waveform frequency domain feature values.

[0086] Furthermore, referring to Figure 2 Step 4 includes:

[0087] Step 41: Obtain the pre-built fault feature mapping library, which contains multiple mapping records, consisting of waveform feature templates and defect feature templates, which respectively contain standard waveform time domain parameter ranges, standard waveform frequency domain parameter ranges, and standard defect image parameter ranges;

[0088] Step 42: Compare the extracted waveform time-domain feature values ​​with the standard waveform time-domain parameter range in each mapping record. If the waveform time-domain feature value falls within the standard waveform time-domain parameter range, a time-domain hit mark is generated; otherwise, a time-domain miss mark is generated.

[0089] Step 43: Compare the extracted waveform frequency domain feature values ​​with the standard waveform frequency domain parameter range in the same mapping record. If the waveform frequency domain feature value falls within the standard waveform frequency domain parameter range, a frequency domain hit mark is generated; otherwise, a frequency domain miss mark is generated.

[0090] Step 44: Compare the extracted defect feature value with the standard defect image parameter range in the same mapping record. If the defect feature value falls within the standard defect image parameter range, a defect hit mark is generated; otherwise, a defect miss mark is generated.

[0091] Step 45: For each mapping record, count the total number of hits of time-domain hit markers, frequency-domain hit markers, and defect hit markers, and determine the mapping record with the highest total number of hits as the candidate matching record;

[0092] Step 46: When the candidate matching record is unique, output the correspondence between the waveform feature template and the defect feature template in the candidate matching record as the matching result, and use the ratio of the total number of hits to the total number of mapped records as the similarity. When the candidate matching record is not unique, compare the interval width of the waveform feature template and the defect feature template in each candidate matching record, select the candidate matching record with the smallest interval width as the matching result, output the correspondence in the matching result, and use the ratio of the total number of hits in the matching result to the total number of mapped records as the similarity.

[0093] In this embodiment, step 41 loads a fault feature mapping library upon processor startup. This library is stored in a table in the database, and its structure includes a mapping record number, a waveform feature template field, and a defect feature template field. In this embodiment, the mapping library contains fifteen records. The waveform feature template in each record consists of seven intervals, corresponding to the fluctuation depth interval, rise rate interval, fall rate interval, fundamental energy percentage interval, second harmonic energy percentage interval, third harmonic energy percentage interval, and total harmonic distortion coefficient interval. Each interval is defined by a minimum and a maximum value. The defect feature template consists of two intervals, corresponding to the texture feature parameter interval and the geometric feature parameter interval. For example, the waveform feature template of the first record is: fluctuation depth interval 80 to 120 amperes, rise rate interval 400 to 600 amperes, and fall rate interval 400 to 600 amperes. Its corresponding defect feature template is: texture feature parameter interval 50 to 80 amperes, and geometric feature parameter interval 2.5 to 3.5 amperes.

[0094] In step 42, the processor sequentially compares the seven waveform time-domain and frequency-domain feature values ​​extracted in step 3 with the seven intervals in the first mapping record. The comparison is performed component by component: it checks if the fluctuation depth is greater than or equal to 80 and less than or equal to 120; if so, the record is marked as a hit in the time domain; otherwise, it is marked as a miss. Then it checks if the rise rate is between 400 and 600, and so on. After comparing the seven components, seven hit marks are obtained; for other mismatched records, only some components may be hit.

[0095] In step 43, the processor compares the four components of the waveform frequency domain feature value—fundamental energy ratio, second harmonic energy ratio, third harmonic energy ratio, and total harmonic energy distortion coefficient—with the standard waveform frequency domain parameter range in the same mapping record to generate a frequency domain hit mark. In actual implementation, the processor can compare all seven components uniformly. In this embodiment, all four frequency domain components are hit, generating four frequency domain hit marks.

[0096] In step 44, the processor compares the texture feature parameters and geometric feature parameters in the extracted defect feature values ​​with the standard defect image parameter range in the same mapping record. It is assumed that the extracted texture feature parameter is 65 and the geometric feature parameter is 3.0.

[0097] In step 45, the processor counts the total number of hits for each mapping record. For the first record, there are seven hits in the time domain and four hits in the frequency domain. Actually, the four hits in the frequency domain are already included in the seven hits in the time domain, but to avoid duplicate counting, counting separately and then adding them would lead to duplication. In actual implementation, only the non-duplicated components should be counted. According to the total number of hits of the time domain hit markers, frequency domain hit markers, and defect hit markers, where the time domain hit markers correspond to seven components and the frequency domain hit markers correspond to four components, the extracted waveform time domain feature values ​​include three values: wave depth, rise rate, and fall rate. The waveform frequency domain feature values ​​include four values: fundamental energy percentage, second harmonic energy percentage, third harmonic energy percentage, and total harmonic energy distortion coefficient, totaling seven. Therefore, there are three time domain hit markers, four frequency domain hit markers, and two defect hit markers, for a maximum of nine hits. In this embodiment, all three time domain hits, all four frequency domain hits, and all two defect hits are achieved, for a total of nine hits. The processor calculates the total number of hits for each of the fifteen records. Assuming the first record has nine hits, the second record has seven hits, the third record has five hits, and the remaining records have four or fewer hits, the first record with the highest total number of hits is determined as the candidate matching record.

[0098] In step 46, since only one record has a total hit count of nine, and all others are less than nine, the candidate matching record is unique. The processor outputs the correspondence between the waveform feature template and the defect feature template in the first record as the matching result. The correspondence is represented as a pointing relationship, indicating that when the waveform feature meets the interval of the first record, the corresponding defect should be insulator damage. The processor also calculates the similarity: the total hit count of nine divided by the total number of mapped records of fifteen, resulting in 0.6, or 60% similarity. If two records have the same total hit count and both are at the highest level, for example, both records hit nine times, the processor compares the interval width of the waveform feature template and the interval width of the defect feature template in these two records. The interval width is calculated as follows: for each record, the upper limit of each of the seven waveform intervals is subtracted from the lower limit to obtain the width value, and the upper limit of each of the two defect intervals is subtracted from the lower limit to obtain the width value. These are summed to obtain the total waveform width and the total defect width. The total waveform width and the total defect width are then added together, and the record with the smaller sum is output as the final candidate matching record.

[0099] Furthermore, referring to Figure 3 Step 5 includes:

[0100] Extract the most similar correspondences from the matching results, including the association pairs between waveform feature values ​​and defect feature values;

[0101] The predefined equipment component mapping table is searched based on the defect feature value in the association pair. The equipment component mapping table records the corresponding entries between the defect feature value and the equipment component name. The faulty equipment identifier is read from the corresponding entry.

[0102] Based on the defect feature values ​​in the associated pair, look up the predefined defect type mapping table and read the fault type code from the defect type mapping table;

[0103] The predefined grade mapping table is searched based on the waveform time-domain feature value and waveform frequency-domain feature value in the association pair. The grade mapping table records the grade values ​​corresponding to different waveform time-domain feature value intervals and waveform frequency-domain feature value intervals. The grade value corresponding to the interval where the waveform time-domain feature value and waveform frequency-domain feature value are located is determined as the fault grade value.

[0104] In this embodiment, the processor extracts the correspondence from the matching results output in step 46. This correspondence is internally represented as a list of key-value pairs, where the key is the identifier of the waveform feature combination and the value is the identifier of the defect feature combination. The processor then obtains the association pair with the highest similarity in this correspondence. Since each candidate matching record is unique, this association pair is the correspondence in the matching results.

[0105] The processor then looks up the device component mapping table based on the defect feature value in the association pair. The device component mapping table is a locally stored SQLite database table named `component_mapping`, containing three fields: defect feature code, device component name, and device identifier. The defect feature code is a hash value calculated from the defect feature value; however, in the specific implementation, the processor directly uses the texture feature parameter range identifier and geometric feature parameter range identifier from the defect feature value as a composite primary key for the query. In this embodiment, the texture parameter range code corresponding to the defect feature value is `TEX_03`, and the geometric parameter range code is `GEO_05`. The processor executes the SQL query statement: `SELECT device_id, component_name FROM component_mapping WHERE tex_code='TEX_03' AND geo_code='GEO_05'`. The query returns one record, where the `device_id` field value is `TOWER_221_B_INSU`, and the `component_name` field value is the B-phase insulator string. The processor uses this `device_id` as the faulty device identifier.

[0106] Then, the processor looks up the defect type mapping table based on the defect feature values ​​in the association pair. The defect type mapping table is also a separate data table named defect_type_mapping, containing two fields: defect feature value range code and fault type code. The defect feature value range code is the same as described above; the processor uses the same combined encoding to look up the code, obtaining the fault type code as DSH_INS_SPL, which indicates insulator damage. This code uses a six-digit alphanumeric combination: the first three digits represent the equipment category, the middle two digits represent the defect type, and the last digit represents the severity level.

[0107] Finally, the processor looks up the level mapping table based on the waveform time-domain and frequency-domain feature values ​​in the association pair. The level mapping table is stored as a multi-dimensional array structure in memory; the first dimension of the array is the fluctuation depth interval index, and the second dimension is the total harmonic energy distortion coefficient interval index. Each array element stores a level value; the processor looks up the element in the level mapping table where the fluctuation depth index (level 3) and the total harmonic energy distortion coefficient index (level 3) intersect, and this element has a value of level 3. Therefore, the fault level value is determined to be level 3.

[0108] Furthermore, the specific method for encapsulating the labeled data packet in step 6 is as follows:

[0109] Create a data container according to the predetermined data exchange format, set the header segment and payload segment in the data container, write the fault candidate evidence collection period into the timestamp field of the header segment, and write the fault device identifier, fault type code and fault level value into the identifier field of the header segment.

[0110] In the load segment, waveform feature blocks and image feature blocks are established. Waveform time-domain feature values ​​and waveform frequency-domain feature values ​​are written into the waveform feature blocks, and defect feature values ​​and key frame images are written into the image feature blocks. The header segment and load segment are combined to form a label data packet, and the label data packet is appended to the end record position of the fault record database.

[0111] In this embodiment, the processor first creates a memory buffer for constructing data packets; the predetermined data exchange format adopts a custom binary protocol, which stipulates that each data packet begins with a fixed four-byte magic word PMUV, followed by a two-byte version number, and then a four-byte header length field.

[0112] The processor begins filling the data container; first, it writes a magic word and version number to the head of the buffer; then it calculates the size of the header segment: the header segment includes a timestamp field, an identifier field, and a reserved field; the timestamp field occupies sixteen bytes, with the first eight bytes representing the Unix seconds of the start time and the last eight bytes representing the nanosecond portion of the start time; similarly, the end time also occupies sixteen bytes. The identifier field occupies thirty-two bytes in total: the fault device identifier is written in UTF-8 encoded string form, occupying a maximum of twenty bytes, padding with zeros if necessary; the fault type code occupies six bytes; the fault level value occupies one byte; the processor converts the start and end timestamps of the fault candidate evidence period into seconds and nanoseconds respectively and writes them into the timestamp field, writes the twenty bytes of the fault device identifier TOWER_221_B_INSU into the device identifier area, writes the six bytes of the fault type code DSH_INS_SPL, and writes the fault level value, encoded as 0x03, into one byte.

[0113] After the header segment is constructed, the processor calculates the total number of bytes in the header segment and writes it to the header length field. Next, the processor begins constructing the payload segment, which uses a TLV type-length-value format. First, the waveform feature block is written: the type field is defined as 0x01, the length field is four bytes, followed by the value field. Seven double-precision floating-point numbers are written sequentially in the value field, each occupying eight bytes, for a total of fifty-six bytes. The processor copies the wave depth, rise rate, fall rate, fundamental energy percentage, second harmonic energy percentage, third harmonic energy percentage, and total harmonic distortion coefficient extracted in step 3 into the value field in sequence. Then, the image feature block is written: the type field is defined as 0x02, the length field is four bytes, and the value field first writes the texture feature parameters and geometric feature parameters from the defect feature values, each occupying four bytes of single-precision floating-point numbers, followed by the keyframe image data. The image data first writes a four-byte integer representing the length of the JPEG data, followed by the JPEG binary data block. The processor reads the JPEG encoded byte array of the keyframe image from memory and copies it to the end of the value field of the image feature block.

[0114] After writing all fields, the processor treats the entire data in the buffer as a complete annotation packet. The processor then opens a connection to the fault record database, which is a sequential record storage system in the form of disk files. Internally, the database maintains a pointer to the current write position, pointing to the end of the last record. The processor appends all bytes of the annotation packet to this position, then updates the database's index file, adding a record to the index that records the packet's starting offset and length within the file. After writing is complete, the database connection is closed.

[0115] Furthermore, the specific method for detecting abnormal fluctuation segments in the waveform data in step 1 is as follows:

[0116] Traverse each sampling point in the waveform data, calculate the amplitude difference between the current sampling point and the previous sampling point, and mark the current sampling point as an abnormal starting point when the absolute value of the amplitude difference exceeds the preset amplitude change threshold;

[0117] Starting from the abnormal starting point, continue to traverse subsequent sampling points and calculate the average amplitude change rate between each subsequent sampling point and the abnormal starting point. When the average amplitude change rate is continuously less than the preset recovery threshold and the duration exceeds the preset stabilization duration, mark the last sampling point that meets the recovery threshold as the abnormal ending point; extract the waveform segment between the abnormal starting point and the abnormal ending point as the abnormal fluctuation segment.

[0118] In this embodiment, the specific method for detecting abnormal fluctuation segments in waveform data in step 1 is as follows: when the processor receives waveform data, it adopts a streaming processing method, and processes each new sampling point immediately upon receipt; the processor maintains a circular buffer with a length of three power frequency cycles to store the most recent historical sampling points.

[0119] The processor iterates through each newly arrived sampling point and compares its amplitude with the amplitude of the previous sampling point in the circular buffer. Let the amplitude of the current sampling point be A_current and the amplitude of the previous sampling point be A_prev. Calculate the difference Δ = A_current - A_prev, and then take the absolute value |Δ|. The processor compares |Δ| with a preset amplitude mutation threshold, which is set to one-third of the normal rated current peak value. For the circuit in this embodiment, the rated current peak value is 1,000 amperes, so the mutation threshold is 333 amperes. When |Δ| is greater than 333 amperes, the processor marks the current sampling point as an abnormal start point and records the sequence number and timestamp of that point.

[0120] Starting from the point of failure, the processor continues to iterate through each subsequent sampling point. Simultaneously, the processor starts a counter to record the number of points that consecutively meet the recovery condition. The recovery condition is defined as follows: the absolute value of the amplitude difference between the current sampling point and the point of failure is less than a preset recovery threshold. The recovery threshold is set to one-tenth of the rated current peak value, i.e., one hundred amperes. For each subsequent sampling point, the processor calculates the absolute value of the difference between the amplitude at that point and the amplitude at the point of failure. If this absolute value is less than one hundred amperes, the counter is incremented; otherwise, the counter is cleared and the iteration continues.

[0121] The processor also maintains a timer for a continuous duration, starting from the first sampling point that meets the recovery conditions. When the counter continuously accumulates to forty, the processor marks the current sampling point that meets the recovery conditions as the end point of the anomaly and stops further detection for the current anomaly segment. If a point that does not meet the recovery conditions occurs before reaching forty points, the counter is reset to zero, the counting restarts, and the anomaly segment continues to extend.

[0122] The processor extracts all sampling points, including the start and end points, between the abnormal start point and the abnormal end point as an abnormal fluctuation segment. If the processor fails to find forty consecutive points that meet the recovery conditions after marking the abnormal start point and continuously traversing for more than one power frequency cycle, it will force the latest sampling point to be the abnormal end point to avoid infinite waiting. The extracted abnormal fluctuation segment, along with the start and end timestamps, is sent to the next step of processing.

[0123] Furthermore, step 3, which involves extracting keyframe images from the video stream and performing device component region detection on the keyframe images to obtain defect feature values, includes:

[0124] Candidate frame images are extracted from the video stream at fixed time intervals. The pixel difference between adjacent candidate frame images is calculated. Candidate frame images whose pixel difference exceeds a preset difference threshold are identified as key frame images. Pixel gradient operations are performed on the key frame images to obtain edge feature maps. The edge feature maps are overlapped and matched with pre-stored device component templates to determine the region where the device component is located.

[0125] Sub-images of the area where the device component is located are cropped from the keyframe image. Texture analysis is performed on the sub-images to extract texture feature parameters. Morphological analysis is performed on the sub-images to extract geometric feature parameters. The texture feature parameters and geometric feature parameters are combined into defect feature values.

[0126] In this embodiment, the specific process of extracting keyframe images from the video stream and performing device component region detection on the keyframe images to obtain defect feature values ​​in step 3 is as follows:

[0127] The processor receives the video stream transmitted back by the drone. The video stream is transmitted in H.264 encoding format with a frame rate of thirty frames per second. The processor first extracts candidate frame images at fixed time intervals, with the time interval set to extract two frames per second, that is, one frame is extracted every fifteen frames. During extraction, the processor decodes the frame image to obtain the original RGB pixel matrix.

[0128] The processor calculates the pixel difference between adjacent candidate frames. To do this, the processor converts two consecutive candidate frames into grayscale images, then calculates the absolute difference of the grayscale values ​​pixel by pixel, summing all the absolute differences to obtain the total difference value. The total difference value is then divided by the total number of pixels in the image to obtain the average pixel difference. When the average pixel difference exceeds a preset difference threshold (set to thirty), the processor designates the next candidate frame as the keyframe image; if the average difference is less than thirty, the frame is skipped, and the next candidate frame is extracted for comparison.

[0129] After obtaining the keyframe image, the processor performs device component region detection on it. First, the processor performs edge extraction on the keyframe image. Specifically, the Sobel operator is used to calculate the gradient in the horizontal and vertical directions respectively to obtain the gradient magnitude of each pixel. Pixels with gradient magnitudes exceeding the set edge threshold, usually fifty, are marked as edge points. Finally, an edge feature map is generated. The pixel values ​​of this map are the same size as the original image, and each pixel value is the edge intensity.

[0130] The processor loads a pre-stored template library of equipment components, containing edge templates for common components such as insulator strings, grading rings, and conductor clamps. Each template is a small-sized binary image; for example, the insulator string template is 100 x 30 pixels, representing the typical outline of an insulator string. The processor performs overlap matching between the edge feature map and each template. The matching process uses a normalized correlation matching method: the template window slides across the edge feature map, the sum of the pixel products of the image and the template within the window is calculated, and the position with the largest sum is taken as the matching position. For the insulator string template, the optimal matching position window has the top-left corner coordinates x=820, y=450, a window width of 100 pixels, and a height of 30 pixels; this window is determined as the region where the insulator string is located.

[0131] The processor crops the region from the original keyframe image to obtain a sub-image with a size of 100 x 30 pixels. Texture analysis is performed on the sub-image: the grayscale values ​​of all pixels in the sub-image are statistically analyzed, the mean grayscale value is calculated, and then the sum of the squares of the differences between the grayscale value of each pixel and the mean is calculated, divided by the total number of pixels to obtain the grayscale variance; this variance is used as a texture feature parameter. Simultaneously, the processor performs morphological analysis on the sub-image: the sub-image is binarized, and then all connected components in the binary image are searched. For each connected component, the width and height of its smallest bounding rectangle are calculated, and the aspect ratio is obtained by dividing the width by the height; the aspect ratio of the connected component with the largest area is selected as a geometric feature parameter. The processor combines the texture feature parameter and the geometric feature parameter into a two-element array, which is output as the defect feature value.

[0132] Furthermore, the mapping records in the fault feature mapping library are pre-established in the following way:

[0133] Collect historical failure cases, each of which includes historical waveform clips of the PMU and historical video clips of the drone;

[0134] Extract waveform time-domain and frequency-domain historical feature values ​​from PMU historical waveform segments, and extract defect historical feature values ​​from UAV historical video segments;

[0135] The waveform time-domain historical feature values ​​and waveform frequency-domain historical feature values ​​are combined into a waveform feature template, and the defect historical feature values ​​are combined into a defect feature template.

[0136] Based on the fault type labeling of historical fault cases, the waveform feature template and defect feature template corresponding to the same fault type are associated and stored as a mapping record;

[0137] For each mapping record, the minimum and maximum values ​​of the waveform time-domain historical feature values ​​in the waveform feature template are taken to form the standard waveform time-domain parameter interval. The minimum and maximum values ​​of the waveform frequency-domain historical feature values ​​are taken to form the standard waveform frequency-domain parameter interval. The minimum and maximum values ​​of the defect historical feature values ​​are taken to form the standard defect image parameter interval.

[0138] In this embodiment, the mapping records in the fault feature mapping library are pre-established in the following manner, and this embodiment describes the complete offline construction process.

[0139] First, historical failure cases were collected. Over the past three years, maintenance personnel recorded 500 field failure cases. Each case included a waveform data file recorded by the PMU device at the time of the failure, as well as video clips taken during the same period by a drone inspection. These cases were all verified by human experts to determine the failure type, the faulty device, and the severity level. These cases were stored in a case library directory, with each case assigned a unique case number.

[0140] For each historical fault case, the processor performs feature extraction. Taking a typical case, case number F-2023-045, as an example: the processor extracts the waveform time-domain historical feature values ​​from the waveform segment: finds local maxima and minima, calculates fluctuation depth, rise rate features, and fall rate features, and then extracts the waveform frequency-domain historical feature values: calculates the fundamental energy ratio, second harmonic energy ratio, third harmonic energy ratio, and total harmonic energy distortion coefficient, obtaining seven values; at the same time, it extracts the defect historical feature values ​​from the drone's historical video segments: extracts keyframes from the video, detects equipment component areas, and obtains texture feature parameters and geometric feature parameters, a total of two values.

[0141] The processor combines these feature values ​​into a template; specifically, it arranges seven values—the waveform time-domain history feature values ​​and the waveform frequency-domain history feature values—in order to form a waveform feature template vector. It also arranges two values ​​of the defect history feature values ​​to form a defect feature template vector. In this case, the waveform feature template is [formula missing], and the defect feature template is [formula missing]: texture feature = 65, geometric feature = 3.0.

[0142] The processor determines the category of the historical fault case based on its fault type label. The manual labeling record shows that the fault type of this case is insulator damage, with the type code INS_BRK. The processor checks whether a mapping record for the same fault type already exists; since this is the first time processing this type, the processor creates a new mapping record, with the record number MAP_023; the waveform feature template and defect feature template are stored as the original templates for this record in the temporary storage area.

[0143] After the processor has processed all 500 cases, each fault type may correspond to multiple cases. For example, there are thirty cases of insulator damage types; the processor performs template merging on all cases under the same fault type; for waveform feature templates, the processor extracts the fluctuation depth values ​​of all thirty cases, finds the minimum value (e.g., 80) and the maximum value (e.g., 120), and forms the standard waveform time domain parameter interval [80, 120]; similarly, the minimum and maximum values ​​of the rise rate feature are taken to obtain the rise rate interval [400, 600]; the interval for the fall rate feature is taken as [380, 550]; the interval for the fundamental frequency energy ratio is taken as [0.90, 0.95]; the interval for the second harmonic energy ratio is taken as [0.02, 0.06]; the interval for the third harmonic energy ratio is taken as [0.01, 0.04]; the interval for the total harmonic energy distortion coefficient is taken as [0.05, 0.11]; for defect feature templates, the minimum value of the texture feature parameter (50) and the maximum value of the texture feature parameter (80) are taken to obtain the interval [50, 80]; the minimum value of the geometric feature parameter (2.5) and the maximum value of the geometric feature parameter (3.5) are taken to obtain the interval [2.5, 3.5]. The processor stores these intervals into the corresponding fields of the MAP_023 record in the fault feature mapping library; if there is only one case for a certain fault type, the upper and lower limits of the interval are both the feature values ​​of that case.

[0144] Ultimately, the fault feature mapping library contains mapping records with the same number of entries as the fault types. Each record provides a standard correspondence range between waveform features and defect features for use during online matching. The processor serializes and stores the completed mapping library to a disk file and loads it into memory when the system starts.

[0145] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0146] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0147] The specific embodiments of the invention have been described in detail above, but they are only examples, and this application is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications or substitutions to the invention are also within the scope of this application. Therefore, all equivalent changes, modifications, and improvements made without departing from the spirit and principles of this application should be covered within the scope of this application.

Claims

1. A fault association annotation method that integrates PMU waveforms and UAV video defects, characterized in that, include: Step 1: Acquire waveform data collected by the PMU device during the operation of the power equipment, detect abnormal fluctuation segments in the waveform data, and mark the time period corresponding to the abnormal fluctuation segment as the fault candidate evidence collection time period; Step 2: Based on the fault candidate evidence collection period and the waveform change pattern of the waveform data in the abnormal fluctuation segment, generate a UAV mission command containing the target device coordinates and gimbal shooting angle, and send the UAV mission command to the UAV control system. Step 3: Receive the video stream collected by the drone at the coordinates of the target device, extract the waveform time-domain feature value and waveform frequency-domain feature value in the abnormal fluctuation segment, extract key frame images from the video stream, and perform device component area detection on the key frame images to obtain defect feature values; Step 4: Input the waveform time-domain feature value, waveform frequency-domain feature value and defect feature value into the feature matcher, and output the matching result according to the pre-established correspondence rules between waveform features and defect features, including the correspondence and similarity between waveform feature values ​​and defect feature values; Step 5: Determine the faulty device identifier, fault type code, and fault level value based on the matching results. The faulty device identifier corresponds to the device component with the highest similarity in the matching results. Step 6: Encapsulate the faulty equipment identifier, fault type code, fault level value, fault candidate evidence collection period, waveform time domain feature value, waveform frequency domain feature value, defect feature value, and key frame image into a labeling data package, and store the labeling data package in the fault record database; Step 4 includes: Step 41: Obtain the pre-built fault feature mapping library, which contains multiple mapping records, consisting of waveform feature templates and defect feature templates, which respectively contain standard waveform time domain parameter ranges, standard waveform frequency domain parameter ranges, and standard defect image parameter ranges; Step 42: Compare the extracted waveform time-domain feature values ​​with the standard waveform time-domain parameter range in each mapping record. If the waveform time-domain feature value falls within the standard waveform time-domain parameter range, a time-domain hit mark is generated; otherwise, a time-domain miss mark is generated. Step 43: Compare the extracted waveform frequency domain feature values ​​with the standard waveform frequency domain parameter range in the same mapping record. If the waveform frequency domain feature value falls within the standard waveform frequency domain parameter range, a frequency domain hit mark is generated; otherwise, a frequency domain miss mark is generated. Step 44: Compare the extracted defect feature value with the standard defect image parameter range in the same mapping record. If the defect feature value falls within the standard defect image parameter range, a defect hit mark is generated; otherwise, a defect miss mark is generated. Step 45: For each mapping record, count the total number of hits of time-domain hit markers, frequency-domain hit markers, and defect hit markers, and determine the mapping record with the highest total number of hits as the candidate matching record; Step 46: When the candidate matching record is unique, output the correspondence between the waveform feature template and the defect feature template in the candidate matching record as the matching result, and use the ratio of the total number of hits to the total number of mapped records as the similarity. When the candidate matching record is not unique, compare the interval width of the waveform feature template and the defect feature template in each candidate matching record, select the candidate matching record with the smallest interval width as the matching result, output the correspondence in the matching result, and use the ratio of the total number of hits in the matching result to the total number of mapped records as the similarity.

2. The method according to claim 1, characterized in that, Step 2 includes: The waveform abrupt change type is identified based on the waveform change pattern within the abnormal fluctuation range of the waveform data. The waveform abrupt change types include voltage sag, voltage swell, harmonic distortion, and pulse spike. For each type of waveform change, a corresponding device coordinate offset and gimbal angle offset are preset; The original equipment installation coordinates of the abnormal waveform are determined based on the fault candidate evidence collection period, and the target equipment coordinates are obtained by superimposing the original equipment installation coordinates with the equipment coordinate offset. The gimbal shooting angle is obtained by adding the gimbal angle offset to the original default angle of the gimbal. The target device coordinates and gimbal shooting angle are encoded into UAV mission commands.

3. The method according to claim 1, characterized in that, Step 3 extracts the waveform time-domain feature values ​​and waveform frequency-domain feature values ​​within the abnormal fluctuation segment, including: Waveform amplitude is sampled sequentially along the time axis of the abnormal fluctuation segment. Local maxima and local minima in the waveform amplitude sequence are recorded. The maximum value in the difference sequence between local maxima and local minima is taken as the fluctuation depth in the waveform time domain feature value. The shortest time required for the waveform amplitude to rise from a local minima to a local maxima is recorded as the rise rate feature in the waveform time domain feature value. The shortest time required for the waveform amplitude to fall from a local maxima to a local minima is recorded as the fall rate feature in the waveform time domain feature value. The waveform amplitude sequence within the abnormal fluctuation segment is decomposed according to a preset frequency window, and the fundamental energy ratio, second harmonic energy ratio, third harmonic energy ratio, and total harmonic energy distortion coefficient are extracted as waveform frequency domain feature values.

4. The method according to claim 1, characterized in that, Step 5 includes: Extract the most similar correspondences from the matching results, including the association pairs between waveform feature values ​​and defect feature values; The predefined equipment component mapping table is searched based on the defect feature value in the association pair. The equipment component mapping table records the corresponding entries between the defect feature value and the equipment component name. The faulty equipment identifier is read from the corresponding entry. Based on the defect feature values ​​in the associated pair, look up the predefined defect type mapping table and read the fault type code from the defect type mapping table; The predefined grade mapping table is searched based on the waveform time-domain feature value and waveform frequency-domain feature value in the association pair. The grade mapping table records the grade values ​​corresponding to different waveform time-domain feature value intervals and waveform frequency-domain feature value intervals. The grade value corresponding to the interval where the waveform time-domain feature value and waveform frequency-domain feature value are located is determined as the fault grade value.

5. The method according to claim 1, characterized in that, The specific method for encapsulating the labeled data packet in step 6 is as follows: Create a data container according to the predetermined data exchange format, set the header segment and payload segment in the data container, write the fault candidate evidence collection period into the timestamp field of the header segment, and write the fault device identifier, fault type code and fault level value into the identifier field of the header segment. In the load segment, waveform feature blocks and image feature blocks are established. Waveform time-domain feature values ​​and waveform frequency-domain feature values ​​are written into the waveform feature blocks, and defect feature values ​​and key frame images are written into the image feature blocks. The header segment and load segment are combined to form a label data packet, and the label data packet is appended to the end record position of the fault record database.

6. The method according to claim 1, characterized in that, The specific method for detecting abnormal fluctuation segments in waveform data in step 1 is as follows: Traverse each sampling point in the waveform data, calculate the amplitude difference between the current sampling point and the previous sampling point, and mark the current sampling point as an abnormal starting point when the absolute value of the amplitude difference exceeds the preset amplitude change threshold; Starting from the abnormal starting point, continue to traverse subsequent sampling points and calculate the average amplitude change rate between each subsequent sampling point and the abnormal starting point. When the average amplitude change rate is continuously less than the preset recovery threshold and the duration exceeds the preset stabilization duration, mark the last sampling point that meets the recovery threshold as the abnormal ending point; extract the waveform segment between the abnormal starting point and the abnormal ending point as the abnormal fluctuation segment.

7. The method according to claim 1, characterized in that, Step 3 involves extracting keyframe images from the video stream and performing device component region detection on the keyframe images to obtain defect feature values, including: Candidate frame images are extracted from the video stream at fixed time intervals. The pixel difference between adjacent candidate frame images is calculated. Candidate frame images whose pixel difference exceeds a preset difference threshold are identified as key frame images. Pixel gradient operations are performed on the key frame images to obtain edge feature maps. The edge feature maps are overlapped and matched with pre-stored device component templates to determine the region where the device component is located. Sub-images of the area where the device component is located are cropped from the keyframe image. Texture analysis is performed on the sub-images to extract texture feature parameters. Morphological analysis is performed on the sub-images to extract geometric feature parameters. The texture feature parameters and geometric feature parameters are combined into defect feature values.

8. The method according to claim 3, characterized in that, The mapping records in the fault feature mapping library are pre-established in the following way: Collect historical failure cases, each of which includes historical waveform clips of the PMU and historical video clips of the drone; Extract waveform time-domain and frequency-domain historical feature values ​​from PMU historical waveform segments, and extract defect historical feature values ​​from UAV historical video segments; The waveform time-domain historical feature values ​​and waveform frequency-domain historical feature values ​​are combined into a waveform feature template, and the defect historical feature values ​​are combined into a defect feature template. Based on the fault type labeling of historical fault cases, the waveform feature template and defect feature template corresponding to the same fault type are associated and stored as a mapping record; For each mapping record, the minimum and maximum values ​​of the waveform time-domain historical feature values ​​in the waveform feature template are taken to form the standard waveform time-domain parameter interval. The minimum and maximum values ​​of the waveform frequency-domain historical feature values ​​are taken to form the standard waveform frequency-domain parameter interval. The minimum and maximum values ​​of the defect historical feature values ​​are taken to form the standard defect image parameter interval.

Images