Equipment abnormity identification method and device based on unmanned aerial vehicle inspection, terminal equipment and storage medium

Abstract

The invention discloses an equipment anomaly recognition method and device based on unmanned aerial vehicle inspection, terminal equipment and a storage medium, and belongs to the technical field of equipment anomaly recognition. The method comprises the steps that equipment ledger data and geographical text data of an inspection area are acquired, and inspection data acquired by an unmanned aerial vehicle are acquired; the data comprise regional temperature data, regional images, unmanned aerial vehicle positioning data and audio data; and extracting a text feature, an audio feature, an unmanned aerial vehicle position feature, a regional temperature distribution feature and an image visual feature from the multi-source data through an equipment state recognition model, and outputting an equipment fault abnormal result through weighted fusion. According to the invention, based on the acquisition of the multi-source data and the association effect among the multi-dimensional features, the equipment abnormity can be accurately identified, early fault hidden dangers can be timely found, and the problems that the fault misjudgment is easily caused due to the one-sidedness of the data acquisition and the fault hidden dangers cannot be timely found in the traditional technology are effectively solved.

Description

Technical Field

[0001] This invention relates to the field of equipment anomaly identification technology, and in particular to a method, apparatus, terminal device and storage medium for equipment anomaly identification based on drone inspection. Background Technology

[0002] By monitoring, analyzing, and judging the operating status data of power equipment, such as transmission towers, transmission lines, transformers, and insulators, it is possible to identify whether there are any faults or anomalies in the equipment and the specific types of faults. This allows for the timely detection of potential defects and early faults in the equipment, enabling early repair measures to be taken and avoiding serious consequences such as equipment damage and large-scale power outages caused by the expansion of faults.

[0003] However, traditional technologies often rely on only a single type of data, such as images or temperature data alone, ignoring the correlation between multi-dimensional data. For example, images alone cannot capture abnormal noises caused by mechanical failures inside the equipment, and temperature data alone is insufficient to detect minute cracks on the equipment's exterior. Ignoring regional geographic text data may lead to missing fault identification caused by environmental factors. Due to the one-sidedness of data collection in existing technologies, a large number of fault features are missed, which can easily lead to missed detection or misjudgment of equipment anomalies and failure to detect early potential faults in a timely manner. Summary of the Invention

[0004] This invention provides a method, apparatus, terminal device, and storage medium for identifying equipment anomalies based on drone inspection. It can collect multi-source data and extract multiple features, thereby accurately identifying equipment anomalies based on the correlation and synergy between multi-dimensional data. It can effectively solve the problems of missed detection and misjudgment caused by the one-sidedness of data collection and the omission of features in the prior art.

[0005] An embodiment of the present invention provides a method for identifying equipment anomalies based on unmanned aerial vehicle (UAV) inspection, comprising:

[0006] Obtain geographic text data and equipment ledger data corresponding to the inspection area; wherein, the inspection area includes: several power equipment;

[0007] Acquire inspection data of the drone within the inspection area; wherein, the inspection data includes: area temperature data, area image, drone positioning data, and audio data collected by the drone;

[0008] The geographic text data, equipment ledger data, and inspection data are input into a preset equipment status recognition model so that the equipment status recognition model can extract text features, audio features, drone location features, regional temperature distribution features, and image visual features.

[0009] The text features, audio features, UAV location features, regional temperature distribution features, and image visual features are weighted and fused to output the fused features; based on the fused features, the device fault and anomaly results are output.

[0010] The fault abnormality results include: several target power devices in abnormal states and the fault type corresponding to each target power device.

[0011] Preferably, the inspection area corresponds to a region location information;

[0012] The extraction process of the text features, audio features, and UAV location features includes:

[0013] Based on the geographic text data, extract the first word vector used to describe the topography or elevation of the region;

[0014] Based on the equipment ledger data, extract the second word vector to describe the type or model of the power equipment;

[0015] Based on the first word vector and the second word vector, text features are generated; wherein, the text features are used to characterize the textual correlation between the semantic information of the geographic environment and the attributes of the device itself;

[0016] Feature extraction is performed on the spectrum of the audio data to generate audio features that characterize the frequency variation trend of the spectrum;

[0017] Based on the UAV positioning data and the area location information, a UAV position feature is generated to characterize the relative distance between the UAV and the inspection area.

[0018] Preferably, the regional temperature data includes: an infrared thermal image corresponding to the inspection area;

[0019] The extraction process of the regional temperature distribution characteristics includes:

[0020] The infrared thermal image is segmented into sub-infrared thermal images corresponding to different devices;

[0021] For each sub-infrared thermal image, extract the temperature numerical characteristics and temperature change trend characteristics of each component of the device;

[0022] Based on the temperature values ​​and temperature change trends of each sub-infrared thermal image, the regional temperature distribution characteristics of the inspection area are generated.

[0023] Preferably, the image visual features include: device appearance features and image environment features;

[0024] The process of extracting the visual features of the image includes:

[0025] Identify the individual electrical devices in the region image;

[0026] Based on the identified power devices, the regional image is divided into several candidate device regions;

[0027] For each device candidate region, extract the texture features, shape features, and color features of the device in the device candidate region;

[0028] Based on the texture features, shape features, and color features corresponding to each device in each candidate device region, generate device appearance features;

[0029] For each candidate device region, extract the vegetation cover features and surface deformation features within the candidate device region.

[0030] Image environment features are generated based on the vegetation coverage and surface deformation characteristics of each candidate area for each device.

[0031] Preferably, the weighted fusion of text features, audio features, UAV location features, regional temperature distribution features, and image visual features to output the fused features includes:

[0032] Based on the audio features and the regional temperature distribution features, a temporal correlation feature is generated; wherein, the temporal correlation feature is used to characterize the temporal correlation between temperature and audio.

[0033] The text features, UAV location features, regional temperature distribution features, device appearance features, and image environment features are spatially mapped to generate spatial correlation features; wherein, the spatial correlation features are used to characterize the spatial correlation between visual features and regional temperature.

[0034] The temporal correlation features, spatial correlation features, text features, audio features, UAV location features, regional temperature distribution features, device appearance features, and image environment features are weighted and fused to output the fused features.

[0035] Preferably, after the output device malfunction result, it further includes:

[0036] For each target power device, a warning message is generated to characterize the existing fault in the target power device; wherein, the warning message includes the fault type corresponding to the target power device.

[0037] Preferably, the generation of the device status recognition model includes:

[0038] Geographic text data samples, equipment ledger data samples, and inspection data samples are used as training samples; wherein, the inspection data samples include: regional temperature data samples, regional image samples, UAV positioning data samples, and audio data samples collected by UAVs.

[0039] Using several training samples and the actual equipment fault anomaly results corresponding to each training sample as input, and the predicted equipment fault anomaly results of each training sample as output, the equipment state recognition model to be trained is iteratively trained until the model converges, generating a preset equipment state recognition model. Based on the above method embodiments, the present invention provides corresponding device embodiments.

[0040] One embodiment of the present invention provides an equipment anomaly identification device based on drone inspection, comprising: a first data acquisition module, a second data acquisition module, and an equipment anomaly result generation module;

[0041] The first data acquisition module is used to acquire geographic text data and equipment ledger data corresponding to the inspection area; wherein, the inspection area includes: a number of power equipment;

[0042] The second data acquisition module is used to acquire inspection data of the UAV within the inspection area; wherein, the inspection data includes: area temperature data, area image, UAV positioning data, and audio data collected by the UAV;

[0043] The equipment anomaly result generation module is used to input the geographic text data, equipment ledger data, and inspection data into a preset equipment status recognition model, so that the equipment status recognition model extracts text features, audio features, UAV location features, regional temperature distribution features, and image visual features; performs weighted fusion of text features, audio features, UAV location features, regional temperature distribution features, and image visual features, and outputs the fused features; and outputs the equipment fault anomaly result based on the fused features.

[0044] The fault abnormality results include: several target power devices in abnormal states and the fault type corresponding to each target power device.

[0045] Based on the above method embodiments, the present invention provides corresponding terminal device embodiments.

[0046] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the device anomaly identification method based on UAV inspection described in the above-described embodiment of the invention.

[0047] Based on the above method embodiments, the present invention provides corresponding storage medium embodiments.

[0048] Another embodiment of the present invention provides a storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the device anomaly identification method based on UAV inspection described in the above embodiment of the invention.

[0049] The following benefits can be obtained by implementing the present invention:

[0050] This invention provides a method, apparatus, terminal device, and storage medium for equipment anomaly identification based on drone inspection. This invention not only acquires equipment ledger data and geographic text data of the inspection area, but also acquires inspection data collected by the drone, including regional temperature data, regional images, drone positioning data, and audio data. This allows for comprehensive data acquisition and analysis of equipment, from textual data to sensory data and spatial location data, overcoming the limitations of traditional technologies with their single data type. Furthermore, this invention uses a pre-set equipment status identification model to extract textual features, audio features, drone location features, regional temperature distribution features, and image visual features from the aforementioned multi-source data. Through weighted fusion, it outputs equipment fault anomaly results based on the fused features, clearly marking the target power equipment with anomalies and the corresponding fault type. Compared with existing technologies, this invention can achieve multi-source data acquisition and multi-feature extraction, thereby enabling accurate identification of equipment anomalies based on the correlation and synergistic effects between multi-dimensional data. This effectively solves the problems of missed detections and misjudgments caused by the one-sidedness of data acquisition and feature omissions in traditional technologies, thus enabling timely detection of early potential faults and preventing the escalation of faults. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating a method for identifying equipment anomalies based on unmanned aerial vehicle (UAV) inspection, provided by an embodiment of the present invention.

[0052] Figure 2 This is a schematic diagram of the structure of an equipment anomaly identification device based on drone inspection provided in an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] like Figure 1 As shown, to address the problem of incomplete data collection in existing technologies, which leads to the omission of numerous fault characteristics and easily results in missed or misjudged equipment anomalies, failing to detect early potential faults in a timely manner, an embodiment of the present invention provides a method for equipment anomaly identification based on unmanned aerial vehicle (UAV) inspection, comprising:

[0055] Step S1: Obtain the geographic text data and equipment ledger data corresponding to the inspection area; wherein, the inspection area includes: several power equipment;

[0056] Indicative geographic text data refers to textual descriptions of the geographic environment of the inspection area, including but not limited to the climate characteristics (such as humidity, extreme temperatures, and precipitation frequency) and topographic conditions (such as mountainous areas, coastal areas, and densely populated urban areas) of the inspection area. Geographic text data can reflect the potential impact of the environment on power equipment, such as the high humidity in coastal areas which can easily lead to equipment corrosion, and the dust in industrial areas which can easily cause insulators to accumulate dirt.

[0057] Equipment ledger data refers to the basic information and historical records of power equipment, including equipment model, installation time, rated parameters (such as rated voltage and power), historical fault records, maintenance records, manufacturer information, etc.

[0058] It is understandable that geographic text data and equipment ledger data are static data, which can provide environmental background and basic equipment information for the analysis of subsequent dynamic inspection data (such as temperature and images), avoiding misjudgments that are out of touch with the actual scenario due to relying solely on dynamic data. For example, the same temperature value has different meanings for equipment in high-temperature regions and cold regions, so it is necessary to combine geographic climate data for judgment.

[0059] Step S2: Obtain inspection data of the UAV within the inspection area; wherein, the inspection data includes: area temperature data, area image, UAV positioning data, and audio data collected by the UAV;

[0060] The illustrative regional temperature data refers to the temperature distribution data of power equipment within the inspection area collected by the UAV using its onboard infrared thermal imaging equipment. This data may include real-time temperature values ​​of various parts of the equipment, temperature gradients (temperature differences between different parts), and localized overheating areas.

[0061] Area images refer to visible light images (such as the appearance of towers, insulators, and conductors) of equipment within an inspection area captured by drones using high-definition cameras. These images can capture physical defects in the equipment, such as broken insulators, broken conductor strands, rusted towers, and foreign objects like bird nests.

[0062] UAV positioning data refers to the real-time location information (such as latitude, longitude, and altitude) of a UAV obtained through positioning systems such as GPS and BeiDou.

[0063] Audio data refers to the sound signals collected by a drone through its microphone during equipment operation, such as the operating noise of a transformer, the corona sound of a wire, and the friction sound of mechanical parts. The sound of normal equipment has a stable frequency and amplitude, while abnormal sounds (such as unusual noises or interference) may reflect internal mechanical faults (such as bearing wear) or electrical faults (such as abnormal corona discharge).

[0064] To illustrate, drones can cover areas difficult for humans to reach (such as high-altitude towers and mountain lines), eliminating blind spots in inspections. Simultaneously, drone inspection efficiency is far higher than manual inspection, enabling rapid and comprehensive data collection over large areas. This invention, therefore, can simultaneously collect four types of dynamic data—temperature, image, location, and audio—using drones to cover multiple dimensions of information, including the thermal state, appearance, location, and sound of equipment, thus overcoming the limitations of traditional data collection methods.

[0065] Step S3: Input the geographic text data, equipment ledger data and inspection data into the preset equipment status recognition model so that the equipment status recognition model can extract text features, audio features, UAV location features, regional temperature distribution features and image visual features;

[0066] The text features, audio features, UAV location features, regional temperature distribution features, and image visual features are weighted and fused to output the fused features; based on the fused features, the device fault and anomaly results are output.

[0067] The fault abnormality results include: several target power devices in abnormal states and the fault type corresponding to each target power device.

[0068] This is illustrative. Equipment status recognition models can process multi-source input data, transforming raw data into analyzable features:

[0069] Text features can be extracted from geographic text data and equipment ledger data. For example, if the geographic text data is of a coastal area and the equipment ledger data is of insulators that have been in operation for 10 years, then text features with high corrosion risk can be extracted.

[0070] Extracting audio features from audio data can describe the frequency peaks or duration of abnormal sounds;

[0071] Extracting the drone's location features from the location data indicates that the current area is at an altitude of 1000 meters and is in a mountainous region;

[0072] Extract regional temperature distribution characteristics from temperature data, such as "the current temperature in the region is 30°C higher than normal";

[0073] Extracting visual features from images, such as the outline of crack edges on the surface of an insulator, may reveal characteristics of physical damage.

[0074] The model performs weighted fusion of the above five types of features, that is, it outputs fused features after comprehensive calculation based on the influence weight of different features on the fault type; finally, based on the fused features, it determines the target power equipment with abnormality and the corresponding fault type. For example, when the target power equipment is tower A, its corresponding fault type is insulator rupture; when the target power equipment is transformer B, its corresponding fault type is joint overheating.

[0075] Therefore, this invention solves the problem of one-sided single-feature analysis in traditional technologies by weighted fusion of multi-dimensional features. By learning the correlation between multiple features and fault types, the probability of missed detections and false alarms can be significantly reduced. Furthermore, it can directly locate abnormal equipment and specific fault types, providing precise guidance for maintenance personnel and shortening fault handling time.

[0076] In a preferred embodiment, for steps S1 and S2, the present invention can mark the distribution area of ​​power grid equipment to obtain multiple different inspection areas.

[0077] For each inspection area, the inspection drone is controlled to inspect the preset inspection area, simultaneously collecting images, videos, and drone positioning data, and obtaining GIS geographic text data and equipment ledger information associated with the area.

[0078] Specifically, drones can be equipped with sensors such as visible light cameras and infrared thermal imagers to perform grid-based inspections within a preset area's coordinate range. When collecting image and video data, the visible light camera captures RGB images of the equipment and its environment at a frequency of 20 frames per second, while the infrared thermal imager simultaneously records temperature field distribution data, forming a visual-thermal infrared dual-modal image stream to obtain regional temperature data and regional images.

[0079] During the flight of the drone, the BeiDou positioning system records positioning information such as latitude, longitude, altitude, and heading angle in real time, with timestamp accuracy down to the millisecond level, ensuring strict alignment between the data and the spatial location.

[0080] The drone is also equipped with a microphone, which can simultaneously record the operating sounds of the equipment (such as low-frequency operating noise of transformers, the sound of circuit breakers operating, and high-frequency noise of discharge) and environmental sounds (such as the sound of vegetation friction and the roar of mudslides), and output the corresponding audio data.

[0081] At the same time, it can automatically retrieve GIS geographic text data of preset areas (such as karst landforms with an altitude of 500-800 meters) and related information in equipment ledgers (such as "the tower model is ZGT2-12, and the service life is 10 years"), thereby obtaining the geographic text data and equipment ledger data corresponding to each inspection area.

[0082] For geographic text data, it is mainly retrieved through the power system's geographic information system (GIS), which includes structured and unstructured text information such as the topography of the inspection area (e.g., "karst landform" "plain"), altitude (e.g., "500-800 meters"), geological features (e.g., "slope 25°, sandstone landform"), and environmental risk labels (e.g., "high-risk landslide area" "flammable shrub area"). It is stored in JSON format and covers the spatial range description of the inspection area, geographic attribute parameters and historical environmental assessment records, providing raw data for the subsequent extraction of the first word vector (topography, altitude features).

[0083] Equipment ledger data can include: equipment type (such as "transformer", "tower", "circuit breaker"), model (such as "ZGT2-12" "S13-M-200 / 10"), specifications, installation location, service life, recent maintenance records, etc., providing a textual basis for extracting the second word vector (equipment type and model features).

[0084] For step S3, in a preferred embodiment, the generation of the device state identification model includes:

[0085] Geographic text data samples, equipment ledger data samples, and inspection data samples are used as training samples; wherein, the inspection data samples include: regional temperature data samples, regional image samples, UAV positioning data samples, and audio data samples collected by UAVs.

[0086] The equipment status recognition model is iteratively trained using several training samples and the actual equipment fault and abnormal results corresponding to each training sample as inputs, and the predicted equipment fault and abnormal results of each training sample as outputs, until the model converges, thus generating the preset equipment status recognition model.

[0087] As an illustration, the training samples encompass geographic text data, equipment ledger data, and inspection data including regional temperature, images, drone positioning, and audio, reflecting the equipment's operating environment, basic information, and real-time status from different perspectives. The combination of multi-source data provides the model with more comprehensive and richer features, reducing the potential limitations of single data types. This allows the model to more accurately capture the characteristics of equipment malfunctions and anomalies, improving the accuracy of predicting equipment malfunction and anomaly outcomes.

[0088] Different types of training samples can cover the state information of equipment under various scenarios and operating conditions. When a model is trained on diverse data, the features and patterns it learns become more universal, enabling it to better adapt to various complex situations that may be encountered in real-world applications, rather than being limited to a specific type of data or scenario. For example, audio data collected by a drone may contain special sounds from abnormal equipment operation. Combined with other data, the model can also achieve good recognition results when faced with the same type of equipment failures under different operating environments and models.

[0089] Based on the trained equipment status recognition model, it can extract text features, audio features, drone location features, regional temperature distribution features, and image visual features from the input geographic text data, equipment ledger data, and inspection data.

[0090] In a preferred embodiment, the inspection area corresponds to a region location information;

[0091] The extraction process of the text features, audio features, and UAV location features includes:

[0092] Based on the geographic text data, extract the first word vector used to describe the topography or elevation of the region;

[0093] Based on the equipment ledger data, extract the second word vector to describe the type or model of the power equipment;

[0094] Based on the first word vector and the second word vector, text features are generated; wherein, the text features are used to characterize the textual correlation between the semantic information of the geographic environment and the attributes of the device itself;

[0095] Feature extraction is performed on the spectrum of the audio data to generate audio features that characterize the frequency variation trend of the spectrum;

[0096] Based on the UAV positioning data and the area location information, a UAV position feature is generated to characterize the relative distance between the UAV and the inspection area.

[0097] In a preferred embodiment, the regional temperature data includes: an infrared thermal image corresponding to the inspection area;

[0098] The extraction process of the regional temperature distribution characteristics includes:

[0099] The infrared thermal image is segmented into sub-infrared thermal images corresponding to different devices;

[0100] For each sub-infrared thermal image, extract the temperature numerical characteristics and temperature change trend characteristics of each component of the device;

[0101] Based on the temperature values ​​and temperature change trends of each sub-infrared thermal image, the regional temperature distribution characteristics of the inspection area are generated.

[0102] In a preferred embodiment, the image visual features include: device appearance features and image environment features;

[0103] The process of extracting the visual features of the image includes:

[0104] Identify the individual electrical devices in the region image;

[0105] Based on the identified power devices, the regional image is divided into several candidate device regions;

[0106] For each device candidate region, extract the texture features, shape features, and color features of the device in the device candidate region;

[0107] Based on the texture features, shape features, and color features corresponding to each device in each candidate device region, generate device appearance features;

[0108] For each candidate device region, extract the vegetation cover features and surface deformation features within the candidate device region.

[0109] Image environment features are generated based on the vegetation coverage and surface deformation characteristics of each candidate area for each device.

[0110] Understandably, by extracting the first word vector describing the terrain / altitude (such as "mountainous area" or "altitude 1000 meters") from geographic text data, and extracting the second word vector describing the equipment type / model (such as "transformer" or "model S11-M-100") from equipment ledger data, and then combining the two to generate text features, the correlation between the geographic environment and equipment attributes can be reflected.

[0111] Specifically, if the geographic text data is "the inspection area is a hilly area with an altitude of 800 meters" and the equipment ledger data is "110kV disconnector, model GW4-126", then the first word vector corresponds to "hilly, 800 meters" and the second word vector corresponds to "disconnector, GW4-126". The text features can characterize the association between hilly terrain (prone to water accumulation) and disconnector (outdoor equipment, requiring corrosion protection).

[0112] Furthermore, spectral analysis is performed on the audio data collected by the drone (such as the sound of the equipment running) to extract frequency change trend characteristics. For example, the audio spectrum of a transformer in normal operation is stable, but high-frequency noise may occur when there is a fault.

[0113] Specifically, the audio spectrum of corona discharge in normal transmission lines is concentrated in the range of 200-500Hz. If high-frequency fluctuations above 1000Hz appear in the audio data and continue to increase, the audio characteristics can capture this trend, indicating that there may be insulation damage in the line.

[0114] Furthermore, by combining the drone's positioning data (such as latitude and longitude) with the location information of the inspection area, the relative distance between the drone and the inspection area is calculated, such as the drone being 50 meters directly above the inspection area.

[0115] Example: If the inspection area is located at 30°N, 120°E, and the drone is positioned at 30°00′01″N, 120°00′01″E, then the relative distance can be converted to "horizontal distance 111 meters, altitude 80 meters". This can be used to determine the validity of the data (such as images and audio) collected by the drone. If the distance is too far, the data may be blurry.

[0116] Furthermore, the infrared thermal image is segmented into sub-images of individual devices (such as "transformer sub-image" and "surge arrester sub-image"), and the temperature values ​​(such as "transformer core temperature 75℃") and trends (such as "temperature rises from 60℃ to 75℃ within 30 minutes") of each component are extracted and finally integrated into the regional temperature distribution characteristics.

[0117] Specifically, in a certain infrared thermal image, the temperature values ​​of the transformer oil tank are "upper oil temperature 65℃, lower oil temperature 58℃", and the temperature change trend is "temperature rise of 5℃ in the past hour". However, the normal temperature of similar equipment should be ≤60℃ and the temperature rise should be slow. Therefore, the final regional temperature distribution characteristics can indicate that the transformer may be overloaded or have an internal fault.

[0118] Furthermore, power equipment is identified from regional images and candidate regions are divided. The texture (e.g., whether the insulator surface is smooth), shape (e.g., whether the tower is tilted), and color (e.g., whether the bushing is carbonized and blackened) features of the equipment are extracted to form the appearance features of the equipment. At the same time, the vegetation coverage and surface deformation around the equipment are extracted to form the image environment features.

[0119] Specifically, in the regional image, the candidate area of ​​"transmission line tower" was identified. Its texture features are "dense rust texture on the tower surface", its shape features are "tower tilt angle of 3°", and its color features are "insulators are grayish-black" (normally they should be white). These constitute the appearance features of the equipment, suggesting that the tower may have aging or structural problems. The vegetation coverage features of this area are "vines entwined around the tower, with a coverage rate of 40%", and the surface deformation features are "ground cracks around the tower foundation with a length of 20cm", which constitute the image environment features.

[0120] Therefore, this invention can extract features closely related to equipment failure. Temperature distribution features can directly reflect abnormal equipment heating (such as short circuits and overloads), equipment appearance features can identify physical damage (such as rust and deformation), and text features can be associated with the impact of geographical environment on equipment (such as equipment in high-altitude areas being susceptible to low temperatures).

[0121] Text features reflect the relationship between the environment and the equipment, audio features capture abnormal operating sounds of the equipment, temperature features monitor heating problems, and visual features identify changes in appearance and the surrounding environment. The model of this invention reduces the risk of misjudgment by a single feature through the extraction and mutual verification of multi-dimensional features. For example, when the transformer temperature rises suddenly, high-frequency noise in the audio and oil leakage on the outside occur at the same time, it indicates that an internal fault has occurred.

[0122] In a preferred embodiment, the weighted fusion of text features, audio features, UAV location features, regional temperature distribution features, and image visual features, and the output of the fused features, includes:

[0123] Based on the audio features and the regional temperature distribution features, a temporal correlation feature is generated; wherein, the temporal correlation feature is used to characterize the temporal correlation between temperature and audio.

[0124] The text features, UAV location features, regional temperature distribution features, device appearance features, and image environment features are spatially mapped to generate spatial correlation features; wherein, the spatial correlation features are used to characterize the spatial correlation between visual features and regional temperature.

[0125] The temporal correlation features, spatial correlation features, text features, audio features, UAV location features, regional temperature distribution features, device appearance features, and image environment features are weighted and fused to output the fused features.

[0126] Specifically, the above process strengthens the intrinsic connection of multimodal features through temporal and spatial correlation, and then integrates all features through weighted fusion to form a comprehensive feature that combines temporal dynamism and spatial consistency, i.e., outputting the fused feature.

[0127] First, by analyzing the synchronous change trends of audio and temperature over time, the dynamic correlation of fault development can be captured, such as whether abnormal sound patterns appear when the temperature rises. Specifically:

[0128] 0-5 minutes: The temperature rises from 75℃ (baseline) to 82℃, the audio spectrum is dominated by low frequencies of 50Hz (normal), and the temporal correlation characteristic value is 0.2 (weak correlation);

[0129] 5-10 minutes: The temperature rises to 92℃, the energy in the 3.2kHz high-frequency band of the audio suddenly increases (discharge sound), and the timing correlation characteristic value is 0.9 (strong correlation).

[0130] Based on the aforementioned audio features and regional temperature distribution features, the final generated temporal correlation feature vector is "[0.2,0.9]", which represents the dynamic trend of "the temperature rises suddenly and the high-frequency sound pattern appears synchronously".

[0131] Secondly, spatial correlation features are further generated:

[0132] By binding the relationship between the environment described by text features and the attributes of the device itself, the drone's location coordinates, the device's temperature distribution, and visual features (appearance / environment) to the same spatial coordinate system, we can first verify the consistency of features in physical location, such as whether the damaged parts detected by vision coincide with the temperature anomaly area. We can also verify the spatial mask matrix of the inspection area and the spatial operation of the drone's positioning data, ensuring that the multimodal features currently being analyzed point to the same device / part, and avoiding misjudgments caused by spatial misalignment.

[0133] Specifically, an example of generating spatial correlation features (taking insulator faults on transmission line towers as an example) is as follows:

[0134] Text features: By describing “tower number G123, the second insulator string on the right is XP-70, and the installation location is 113°25′E, 23°18′N” using the first and second word vectors, the spatial semantics of “second insulator string on the right + specific coordinates” can be derived.

[0135] UAV location characteristics: Beidou positioning data shows that the current latitude and longitude of the UAV is (113°25′E, 23°18′N), the deviation from the tower coordinates is less than 3 meters, the shooting distance is 15 meters, and the heading angle is 90° (facing the right side of the tower). The calculated location characteristic semantics are "the relative distance between the UAV and the target insulator is 15 meters, and the spatial matching degree is 0.95".

[0136] Regional temperature distribution characteristics: After segmenting the tower into sub-images from the infrared thermal image, the temperature matrix of the second string of insulators on the right was extracted, showing that the temperature of 3 insulators reached 48℃ (32℃ for other insulators in the same string, and 30℃ for the historical baseline). The pixel coordinates of the temperature anomaly area correspond to the physical location of "upper middle part of the second string on the right".

[0137] Equipment appearance characteristics: The YOLOv8 model can identify and divide the candidate region of "the second string of insulators on the right". The three insulators in this region have visual features of "crack texture (feature value 0.85)" and "black color (feature value 0.78)". The pixel coordinates overlap with the infrared temperature anomaly area by more than 90%.

[0138] Image environment features: The candidate area is free from vegetation obstruction (vegetation coverage feature value 0.05) and the ground surface shows no obvious deformation (deformation feature value 0.03), which can be used to exclude the influence of environmental interference on feature analysis.

[0139] Spatial mapping and consistency verification:

[0140] The spatial information of the above features (textual description semantics, UAV positioning coordinates, infrared / visual pixel coordinates) is uniformly transformed to the UTM coordinate system, and spatial overlay analysis is performed using the PostGIS database:

[0141] The coordinates of the "second insulator string on the right" in the text feature deviate from the coordinates of the "center of the field of view" in the UAV positioning by less than 0.5 meters;

[0142] The spatial range of the infrared temperature anomaly area (48℃) completely overlaps with the "crack + blackening" area detected by visual inspection.

[0143] All features fall within the "tower G123" spatial mask matrix marked by the satellite image (i.e., the mask matrix of the inspection area);

[0144] Therefore, the consistency of the features in physical location has been verified at this point, and all data and images are valid. Then, the generation of spatial correlation features can be carried out.

[0145] Output spatial correlation features:

[0146] Based on the above indicators such as spatial matching degree, coordinate deviation, and regional overlap, the spatial association feature vector is generated as "[0.95,0.98,1.0]" (corresponding to text-location matching degree, infrared-visual region overlap, and proportion within the mask matrix, respectively). Finally, the spatial association score of "0.97" is obtained through weighted calculation.

[0147] This invention can avoid the spatial misalignment problem of incorrectly associating the temperature anomaly of component A with the visual damage of component B through superposition analysis under a unified coordinate system, and provides a reliable spatial reference for subsequent equipment anomaly identification.

[0148] Furthermore, after deriving the temporal and spatial correlation features, weights are assigned to the temporal correlation features, spatial correlation features, text features, audio features, UAV location features, regional temperature distribution features, equipment appearance features, and image environment features. These weighted sums are then used to generate the fused features.

[0149] If the weighted calculation of each feature yields a comprehensive score of 0.8744, then the fused feature score is 0.8744. If the anomaly threshold is 0.7, then it is determined to be an anomaly, and the output fault result is: The target power equipment with an abnormal state is: the insulator of the transmission line tower; the corresponding fault type is: insulator damage with partial discharge.

[0150] It is understandable that the criteria for determining the above-mentioned fault types can be as follows:

[0151] The regional temperature distribution characteristics are as follows: the temperature of the upper three insulators in the second string on the right reaches 48℃, which exceeds the historical baseline (30℃) by 60%, consistent with the temperature rise characteristics caused by partial discharge;

[0152] The temporal correlation feature is represented as follows: during the temperature rise process, a 3-5kHz high-frequency discharge sound pattern (audio feature) was detected simultaneously, and the two were strongly correlated (feature value 0.9), which confirms the dynamic development of the discharge fault.

[0153] Spatial correlation features are represented as follows: the temperature anomaly area, the visual damage area (equipment appearance features) completely overlaps with the text description location (spatial correlation score 0.97), confirming that the faulty component is "the upper middle part of the second string of insulators on the right";

[0154] The equipment's appearance is characterized by cracked textures (characteristic value 0.85) and blackened color (characteristic value 0.78) on the insulators in this area, providing visual evidence of damage.

[0155] Therefore, by weighting and summing the temporal correlation features, spatial correlation features, text features, audio features, UAV location features, regional temperature distribution features, equipment appearance features, and image environment features to generate the fused features, it can be concluded that the fault type should be insulator damage accompanied by partial discharge.

[0156] In a preferred embodiment, after the output device malfunction result, the method further includes:

[0157] For each target power device, a warning message is generated to characterize the existing fault in the target power device; wherein, the warning message includes the fault type corresponding to the target power device.

[0158] In illustrative terms, this invention collects multimodal inspection data, including images, infrared, and audio, using drones and synchronizing it with GIS positioning. Then, it can use models such as YOLOv8 and BERT (i.e., equipment status recognition models) to extract and fuse visual, textual, and audio features, identify abnormal equipment, and provide early warnings of potential fault risks for each piece of equipment. It can also generate a comprehensive risk heat map to guide operation and maintenance. This invention improves the accuracy of equipment identification and the efficiency of operation and maintenance decisions in complex power scenarios, providing a highly reliable and adaptable full-scenario solution for intelligent operation and maintenance of the power grid.

[0159] like Figure 2 As shown, based on the above embodiments of various equipment anomaly identification methods based on UAV inspection, the present invention provides corresponding device embodiments;

[0160] One embodiment of the present invention provides an equipment anomaly identification device based on drone inspection, comprising: a first data acquisition module, a second data acquisition module, and an equipment anomaly result generation module;

[0161] The first data acquisition module is used to acquire geographic text data and equipment ledger data corresponding to the inspection area; wherein, the inspection area includes: a number of power equipment;

[0162] The second data acquisition module is used to acquire inspection data of the UAV within the inspection area; wherein, the inspection data includes: area temperature data, area image, UAV positioning data, and audio data collected by the UAV;

[0163] The equipment anomaly result generation module is used to input the geographic text data, equipment ledger data, and inspection data into a preset equipment status recognition model, so that the equipment status recognition model extracts text features, audio features, UAV location features, regional temperature distribution features, and image visual features; performs weighted fusion of text features, audio features, UAV location features, regional temperature distribution features, and image visual features, and outputs the fused features; and outputs the equipment fault anomaly result based on the fused features.

[0164] The fault abnormality results include: several target power devices in abnormal states and the fault type corresponding to each target power device.

[0165] It should be noted that the device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0166] Those skilled in the art will understand that, for convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0167] Based on the above embodiments of various equipment anomaly identification methods based on UAV inspection, the present invention provides corresponding embodiments for terminal devices.

[0168] One embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a device anomaly identification method based on UAV inspection as described in any embodiment of the present invention.

[0169] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0170] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting various parts of the terminal device via various interfaces and lines.

[0171] The memory can be used to store the computer program. The processor implements various functions of the terminal device by running or executing the computer program stored in the memory and calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, SmartMediaCard (SMC), Secure Digital (SD) card, FlashCard, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0172] Based on the above embodiments of various equipment anomaly identification methods based on UAV inspection, the present invention provides corresponding embodiments of storage media.

[0173] One embodiment of the present invention provides a storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute a device anomaly identification method based on UAV inspection as described in any embodiment of the present invention.

[0174] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0175] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for identifying equipment anomalies based on unmanned aerial vehicle (UAV) inspection, characterized in that, include: Obtain geographic text data and equipment ledger data corresponding to the inspection area; wherein, the inspection area includes: several power equipment; Acquire inspection data of the drone within the inspection area; wherein, the inspection data includes: area temperature data, area image, drone positioning data, and audio data collected by the drone; The geographic text data, equipment ledger data, and inspection data are input into a preset equipment status recognition model so that the equipment status recognition model can extract text features, audio features, drone location features, regional temperature distribution features, and image visual features. The text features, audio features, UAV location features, regional temperature distribution features, and image visual features are weighted and fused to output the fused features; based on the fused features, the device fault and anomaly results are output. The fault abnormality results include: several target power devices in abnormal states and the fault type corresponding to each target power device.

2. The equipment anomaly identification method based on UAV inspection as described in claim 1, characterized in that, The inspection area corresponds to a region location information; The extraction process of the text features, audio features, and UAV location features includes: Based on the geographic text data, extract the first word vector used to describe the topography or elevation of the region; Based on the equipment ledger data, extract the second word vector to describe the type or model of the power equipment; Based on the first word vector and the second word vector, text features are generated; wherein, the text features are used to characterize the textual correlation between the semantic information of the geographic environment and the attributes of the device itself; Feature extraction is performed on the spectrum of the audio data to generate audio features that characterize the frequency variation trend of the spectrum; Based on the UAV positioning data and the area location information, a UAV position feature is generated to characterize the relative distance between the UAV and the inspection area.

3. The equipment anomaly identification method based on UAV inspection as described in claim 2, characterized in that, The regional temperature data includes: infrared thermal images of the inspection area; The extraction process of the regional temperature distribution characteristics includes: The infrared thermal image is segmented into sub-infrared thermal images corresponding to different devices; For each sub-infrared thermal image, extract the temperature numerical characteristics and temperature change trend characteristics of each component of the device; Based on the temperature values ​​and temperature change trends of each sub-infrared thermal image, the regional temperature distribution characteristics of the inspection area are generated.

4. The equipment anomaly identification method based on UAV inspection as described in claim 3, characterized in that, The image visual features include: device appearance features and image environment features; The process of extracting the visual features of the image includes: Identify the individual electrical devices in the region image; Based on the identified power devices, the regional image is divided into several candidate device regions; For each device candidate region, extract the texture features, shape features, and color features of the device in the device candidate region; Based on the texture features, shape features, and color features corresponding to each device in each candidate device region, generate device appearance features; For each candidate device region, extract the vegetation cover features and surface deformation features within the candidate device region. Image environment features are generated based on the vegetation coverage and surface deformation characteristics of each candidate area for each device.

5. The equipment anomaly identification method based on UAV inspection as described in claim 4, characterized in that, The weighted fusion of text features, audio features, UAV location features, regional temperature distribution features, and image visual features, and the output of the fused features, include: Based on the audio features and the regional temperature distribution features, a temporal correlation feature is generated; wherein, the temporal correlation feature is used to characterize the temporal correlation between temperature and audio. The text features, UAV location features, regional temperature distribution features, device appearance features, and image environment features are spatially mapped to generate spatial correlation features; wherein, the spatial correlation features are used to characterize the spatial correlation between visual features and regional temperature. The temporal correlation features, spatial correlation features, text features, audio features, UAV location features, regional temperature distribution features, device appearance features, and image environment features are weighted and fused to output the fused features.

6. The equipment anomaly identification method based on UAV inspection as described in claim 5, characterized in that, Following the output device malfunction / abnormal result, it also includes: For each target power device, a warning message is generated to characterize the existing fault in the target power device; wherein, the warning message includes the fault type corresponding to the target power device.

7. The equipment anomaly identification method based on UAV inspection as described in claim 1, characterized in that, The generation of the device status recognition model includes: Geographic text data samples, equipment ledger data samples, and inspection data samples are used as training samples; wherein, the inspection data samples include: regional temperature data samples, regional image samples, UAV positioning data samples, and audio data samples collected by UAVs. The equipment status recognition model is iteratively trained using several training samples and the actual equipment fault and abnormal results corresponding to each training sample as inputs, and the predicted equipment fault and abnormal results of each training sample as outputs, until the model converges, thus generating the preset equipment status recognition model.

8. A device for identifying equipment anomalies based on unmanned aerial vehicle (UAV) inspection, characterized in that, include: The system comprises a first data acquisition module, a second data acquisition module, and a device anomaly result generation module. The first data acquisition module is used to acquire geographic text data and equipment ledger data corresponding to the inspection area; wherein, the inspection area includes: a number of power equipment; The second data acquisition module is used to acquire inspection data of the UAV within the inspection area; wherein, the inspection data includes: area temperature data, area image, UAV positioning data, and audio data collected by the UAV; The equipment anomaly result generation module is used to input the geographic text data, equipment ledger data, and inspection data into a preset equipment status recognition model, so that the equipment status recognition model extracts text features, audio features, UAV location features, regional temperature distribution features, and image visual features; performs weighted fusion of text features, audio features, UAV location features, regional temperature distribution features, and image visual features, and outputs the fused features; and outputs the equipment fault anomaly result based on the fused features. The fault abnormality results include: several target power devices in abnormal states and the fault type corresponding to each target power device.

9. A terminal device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement a device anomaly identification method based on unmanned aerial vehicle (UAV) inspection as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium includes a stored computer program, wherein, when the computer program is running, it controls the device where the storage medium is located to execute a device anomaly identification method based on UAV inspection as described in any one of claims 1 to 7.

Images