Data processing method and system for unmanned aerial vehicle to inspect insulators, electronic device and medium

Abstract

The present application relates to the field of power system maintenance, and particularly relates to a data processing method and system for unmanned aerial vehicle (UAV) inspection of insulators, an electronic device and a medium; the method comprises: acquiring multi-physical field sensing data stream and multi-angle image data stream of the insulator, to perform signal preprocessing and feature encoding, and output an electric field distribution feature vector; generating a multi-dimensional fusion feature representation through feature-level fusion; performing inference on the multi-dimensional fusion feature representation, to output a recognition result of an insulator surface abnormality category and a spatial coordinate of an abnormal position; performing state grading through a dynamic threshold evaluation model, to generate a structured insulator comprehensive state report. In this way, the technical problem of insufficient accuracy of evaluation of the overall operating state of the insulator in the prior art is solved, and the discovery capability of multi-factor coupling faults, the reliability of state evaluation and the effectiveness of UAV inspection in supporting intelligent operation and maintenance decision of the power grid are improved.

Description

Technical Field

[0001] This invention relates to the field of power system maintenance, and in particular to a data processing method, system, electronic device, and medium for unmanned aerial vehicle (UAV) inspection of insulators. Background Technology

[0002] In power systems, insulators are critical equipment ensuring the safe and stable operation of transmission lines. Accurate assessment and intelligent diagnosis of their operational status are essential for preventing power grid failures and improving maintenance efficiency. With the deep integration and rapid development of smart grids and drone technology, using drones for transmission line inspection has become a mainstream trend, replacing traditional, inefficient, and high-risk manual inspection methods. Drones, with their superior mobility, wide coverage, and efficient operational capabilities, provide an advanced technological platform for the periodic and large-scale inspection of insulator conditions. However, insulators are exposed to complex natural and electromagnetic environments for extended periods, resulting in diverse forms of degradation, including but not limited to surface contamination accumulation, internal or surface discharge, composite material aging, and mechanical damage. These defects often exhibit characteristics of multi-physics coupling. Therefore, relying solely on single-type or limited-dimensional sensing data is insufficient to comprehensively and accurately depict the overall health status of insulators, placing higher demands on the processing and analysis capabilities of drone inspection data.

[0003] In the field of intelligent drone inspection of insulators, existing technologies have undergone numerous explorations, but they generally suffer from limitations in data dimensions and a single evaluation perspective. Existing technology one (patent publication number CN116958036A) discloses a drone inspection method based on hyperspectral technology, which assesses the contamination status by analyzing the hyperspectral image features of the insulator surface. While this method is effective for contamination detection, it relies solely on hyperspectral images, failing to effectively detect faults unrelated to surface contamination, such as abnormal electric field distribution or localized overheating, resulting in insufficient assessment of the overall insulator condition. Existing technology two (application publication number CN116665080A) discloses a drone-based method for detecting deteriorated insulators based on target recognition. It uses an improved YOLOv8 (You Only Look Once version 8) model to perform instance segmentation and target detection on insulator infrared images to identify deteriorated areas. This method focuses on defect detection in infrared images, improving recognition accuracy in specific scenarios, but its technical solution is also limited to the single data dimension of infrared images. Some early deterioration or non-thermal-related faults in insulators (such as local electric field distortion and internal moisture) may not be clearly characterized in infrared images, leading to the risk of missed detections or misjudgments in this method. Another patent (publication number CN110967600A) also mainly relies on UAV infrared detection data for composite insulator deterioration diagnosis, and its diagnostic dimensions are also relatively limited. In addition, although other related studies have optimized image recognition algorithms to improve defect detection accuracy, they have not fundamentally broken through the dependence on single-mode data such as visible light or infrared. Therefore, the common prominent problem of existing UAV-based insulator inspection technologies, whether relying on hyperspectral images, infrared images, or visible light images, is that due to the reliance on single-dimensional or limited types of data, the assessment of the overall operating status of the insulator is not comprehensive or accurate enough, and it is difficult to cover the various potential fault types caused by the coupling of multiple factors such as electric field, thermal field, and mechanical stress. Summary of the Invention

[0004] To address the aforementioned shortcomings or deficiencies, this invention provides a data processing method, system, electronic device, and medium for drone-based insulator inspection, which can solve the technical problem of insufficient accuracy in assessing the overall operating status of insulators in existing drone-based insulator inspection technologies.

[0005] This invention provides a data processing method for unmanned aerial vehicle (UAV) inspection of insulators, comprising: Perform multi-dimensional data synchronous acquisition and spatiotemporal alignment to obtain multi-physics field sensing data streams and multi-angle image data streams of insulators.

[0006] The airborne edge computing node performs signal preprocessing and feature encoding on the electric field measurement data in the multiphysics sensing data stream, and outputs the electric field distribution feature vector.

[0007] At the airborne edge computing node, the electric field distribution feature vector, non-electric field modal data from the multi-physics sensing data stream, and multi-angle image data stream are input into the cross-modal feature fusion network. Through feature-level fusion based on attention weights, a unified multi-dimensional fusion feature representation is generated.

[0008] The multi-dimensional fused feature representation is transmitted to the ground computing node. The lightweight target detection model deployed on the ground computing node is used to infer the multi-dimensional fused feature representation and output the identification result of the anomaly category on the insulator surface and the spatial coordinates of the anomaly location.

[0009] Based on the identification results and the spatial coordinates of the abnormal locations, combined with the insulator's operating context features, a structured insulator comprehensive status report is generated by evaluating the execution status classification of the dynamic threshold assessment model.

[0010] According to a second aspect, the present invention provides a data processing system for unmanned aerial vehicle (UAV) inspection of insulators, comprising: The data stream acquisition module is used to perform multi-dimensional data synchronous acquisition and spatiotemporal alignment, and to acquire the multi-physics field sensing data stream and multi-angle image data stream of the insulator.

[0011] The feature vector output module is used to perform signal preprocessing and feature encoding on electric field measurement data in multiphysics sensing data streams at airborne edge computing nodes, and output electric field distribution feature vectors.

[0012] The feature representation generation module is used to input electric field distribution feature vectors, non-electric field modal data from multi-physics sensing data streams, and multi-angle image data streams into a cross-modal feature fusion network at airborne edge computing nodes. Through feature-level fusion based on attention weights, a unified multi-dimensional fused feature representation is generated.

[0013] The spatial coordinate output module is used to transmit the multi-dimensional fused feature representation to the ground computing node. It uses a lightweight target detection model deployed on the ground computing node to infer the multi-dimensional fused feature representation and outputs the identification results of the anomaly category on the insulator surface and the spatial coordinates of the anomaly location.

[0014] The status report generation module is used to generate a comprehensive status report for structured insulators by performing status grading through a dynamic threshold evaluation model based on the identification results and the spatial coordinates of the abnormal locations, combined with the insulator operating context features.

[0015] According to a third aspect, the present invention provides an electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform any of the data processing methods for inspecting insulators by drones in the embodiments of the present invention.

[0016] According to another aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute a data processing method for inspecting insulators by any UAV in the embodiments of the present invention.

[0017] The present invention provides a data processing method for UAV inspection of insulators. This method is achieved through four core stages: multi-dimensional data synchronous acquisition and spatiotemporal alignment, signal processing and feature fusion of airborne edge computing nodes, model inference and dynamic threshold evaluation of ground computing nodes. The process involves simultaneous multi-dimensional data acquisition and spatiotemporal alignment to obtain multi-physics sensing data streams and multi-angle image data streams for insulators. This integrates electric field, thermal, humidity, and multi-view visual information from the data source, laying a multi-dimensional data foundation for comprehensive state perception. At the airborne edge computing node, signal preprocessing and feature encoding of the electric field measurement data are performed to output an electric field distribution feature vector, achieving deep analysis and feature extraction of this key latent state quantity. At the same node, the electric field distribution feature vector, non-electric field modal data, and multi-angle image data streams are input to a cross-modal feature fusion network. A unified multi-dimensional fusion feature representation is generated through attention-weighted feature-level fusion, constructing a comprehensive state representation that reflects the correlation of multiple physical fields. This multi-dimensional fusion feature representation is transmitted to a ground computing node and inferred using a lightweight target detection model, outputting the identification results and spatial coordinates of insulator surface anomalies, achieving accurate location and classification of potential defects. Based on the identification results and spatial coordinates, combined with the insulator operating context features, a dynamic threshold evaluation model is used to perform state grading to generate a structured comprehensive state report, achieving adaptive matching between the evaluation conclusions and the actual operating conditions of the equipment.

[0018] In this technical solution, the present invention addresses the technical problem described in the background art, which is that the assessment of the overall operating status of insulators is not comprehensive and accurate enough due to reliance on single or limited-dimensional data. By introducing a step of synchronous acquisition and fusion of multi-dimensional data, it fundamentally breaks through the limitations of a single data dimension, enabling the assessment of insulators to simultaneously cover multiple aspects such as electric field, thermal field, mechanical, and visual information, thus solving the fundamental defect of the one-sided assessment perspective in existing technologies. To address the problems of inaccurate assessment results and insufficient ability to detect faults without significant specific modal characteristics caused by this defect, the present invention establishes a process that combines "edge-side multi-modal feature fusion" with "cloud-based lightweight recognition," and finally uses a "dynamic threshold model" for hierarchical assessment. First, a cross-modal fusion network is used to generate a unified feature representation that reflects the inherent correlation of multi-source information, providing a more complete input for the recognition model. Then, dynamic discrimination is performed in conjunction with the entire life cycle context of the equipment, thereby achieving a comprehensive and accurate assessment of the complex and potential operating states of insulators in terms of data processing logic and decision-making mechanism. Therefore, the technical solution of the present invention solves the technical problem of insufficient accuracy in assessing the overall operating status of insulators in the prior art, and improves the ability to detect multi-factor coupled faults, the reliability of status assessment, and the effectiveness of UAV inspection in supporting intelligent operation and maintenance decision-making of power grid. Attached Figure Description

[0019] Figure 1 This is a flowchart of a data processing method for unmanned aerial vehicle (UAV) inspection of insulators according to an embodiment of the present invention; Figure 2 This diagram illustrates the specific hardware and software functional modules and data flow of a multiphysics sensing and intelligent analysis system according to another embodiment of the present invention. Figure 3 This is a schematic diagram illustrating the process of collaborative processing of multi-physics measurement data to generate electric field distribution characteristic parameters according to another embodiment of the present invention. Figure 4 This is a schematic diagram illustrating the multiphysics feature data processing and fusion process according to another embodiment of the present invention; Figure 5 This diagram illustrates the complete training and deployment process of an improved lightweight YOLO object detection model according to another embodiment of the present invention. Figure 6 This is a schematic diagram of the data processing system for unmanned aerial vehicle (UAV) inspection of insulators according to an embodiment of the present invention; Figure 7 This is a block diagram of an electronic device used to implement embodiments of the present invention. Detailed Implementation

[0020] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] During the development of this invention, researchers conducted numerous experiments and data analyses, revealing the intrinsic relationship between different physical states of insulators (such as electric field distribution, surface temperature, ambient humidity, and visual appearance) and their overall operational health. Single-dimensional sensing data can only reflect local characteristics of the state, while multi-physics data exhibit strong coupling and complementarity, and their collaborative change patterns are key to accurately identifying complex and potential defects. Based on this relationship, this invention innovatively proposes this technical solution. Utilizing a multi-physics sensing array and visual sensors integrated into an unmanned aerial vehicle (UAV) platform, it performs signal preprocessing, feature encoding, and attention-weighted cross-modal fusion at the airborne edge computing node. Combined with lightweight model inference and dynamic threshold-based hierarchical evaluation at the ground computing node, it achieves fully automated and precise processing of the entire process from synchronous multi-dimensional data acquisition to intelligent state hierarchical evaluation of insulators. This embodies the core concept of "deep fusion of multi-modal information and adaptive decision-making under cloud-edge-device collaborative computing."

[0022] Specifically, through comparative experiments, the invention team discovered that traditional insulator condition assessment methods based on a single type of data (such as visible light images or infrared images only) suffer from technical defects such as "one-sided assessment dimensions and incomplete feature perception." Their analytical models can only capture and respond to anomalies in specific physical fields (such as thermal radiation or specific spectral reflection), lacking the ability to perceive faults not explicitly characterized within this data dimension (such as electric field distortion caused by early partial discharge without significant temperature rise). These technical defects lead to insufficient reliability and comprehensiveness of the assessment results, making it prone to missed detections and difficult to support preventative maintenance decisions. The processing method based on synchronous acquisition and fusion of multi-dimensional data proposed in this invention can improve the completeness of state perception and the richness of feature representation. By performing real-time processing and cross-modal fusion of multi-physical field data such as electric field at the edge, it is possible to achieve early extraction of latent fault features and complementary enhancement of multi-source information. By using a lightweight target detection model to reason about a unified multi-dimensional fused feature representation, it is possible to ensure the recognition accuracy and positioning accuracy of multiple anomaly categories in complex backgrounds and small target situations. By combining a dynamic threshold evaluation model with operating context features for state classification, it is possible to ensure the adaptability of evaluation conclusions to the actual working conditions and historical states of insulators, and output a structured report that can directly guide operation and maintenance actions.

[0023] Therefore, this invention provides a data processing method for insulator inspection by unmanned aerial vehicles (UAVs), which can be applied to a cloud-edge-device collaborative intelligent insulator inspection and condition assessment system (hereinafter referred to as the "system"). This system can operate in a fully automatic or semi-autonomous mode in a distributed computing environment that coordinates cloud, edge, and terminal devices to complete the entire process from multi-dimensional data acquisition, fusion analysis, defect identification to intelligent condition assessment of insulators. Specifically, this system can be deployed in various hardware environments, including but not limited to: UAV platforms equipped with multi-physics field sensor arrays and computing units (edge), ground control stations or cloud server clusters (cloud), and maintenance terminals for result display and decision support (terminals). This flexible deployment architecture allows the system to meet both the edge computing requirements for real-time data preprocessing and low-latency response at UAV inspection sites and the cloud computing requirements for high-precision, large-scale inference of complex models.

[0024] like Figure 1 As shown, the method may include: Step S110: Perform multi-dimensional data synchronous acquisition and spatiotemporal alignment to obtain the multi-physics field sensing data stream and multi-angle image data stream of the insulator.

[0025] Among them, a multi-physics field sensor array refers to a combination of sensors integrated and installed on a drone platform for synchronously measuring various physical quantities such as electric field, infrared thermal radiation, and micro-environment temperature and humidity around an insulator; a high-definition vision sensor refers to a camera device with high resolution and support for controllable attitude shooting; a multi-physics field sensing data stream refers to a time-series data set that is continuous in time and synchronized, output by a multi-physics field sensor array; and a multi-angle image data stream refers to a spatially correlated sequence of images captured by a high-definition vision sensor from different observation angles.

[0026] Specifically, the system can coordinate a multiphysics sensor array and a high-definition visual sensor through the UAV flight control system to synchronously trigger data acquisition at inspection track points. The acquired raw data is appended with a unified timestamp and spatial coordinate label provided by the Global Navigation Satellite System (GNSS) and a high-precision clock source, and transmitted to the buffer via the airborne data bus. For example, the multiphysics sensor array used by the system includes a high-sensitivity fiber optic electric field sensor with a sensitivity of 10 volts per meter (V / m) and a thermal sensitivity sensor with a sensitivity of 0.05 degrees Celsius (V / m). The system employs an infrared thermal imaging sensor and an integrated temperature and humidity sensor. The high-definition vision sensor is a 4K resolution industrial camera. During a single inspection mission, the system hovers and collects data at 10 preset track points. At each point, it simultaneously acquires one set of electric field data (sampling frequency 100 kHz, duration 1 second), one frame of infrared thermal image (resolution 640×512 pixels), and five visible light images from different angles (resolution 3840×2160 pixels). All data is accompanied by timestamps accurate to milliseconds (ms) and spatial coordinates with centimeter (cm) precision, collectively forming the initial sensing data stream and image stream.

[0027] Step S120: Perform signal preprocessing and feature encoding on the electric field measurement data in the multiphysics sensing data stream at the airborne edge computing node, and output the electric field distribution feature vector.

[0028] Signal preprocessing refers to the operation of denoising and transforming the original electric field signal to improve signal quality and analyzability; feature encoding refers to the process of extracting representative low-dimensional mathematical representations from the preprocessed signal; electric field distribution feature vector refers to a fixed-dimensional real number vector used to quantify the distortion, gradient and phase distribution characteristics of the electric field around the insulator.

[0029] Specifically, the system can first perform adaptive Kalman filtering on the electric field measurement data stream to suppress random noise through a data processing program running on an airborne edge computing node. Then, it performs multi-scale time-frequency analysis based on wavelet packet transform on the denoised time-domain signal to resolve the signal components in different frequency bands. Next, it performs local contrast enhancement on the signal components to strengthen the features of potential distortion regions. Finally, it maps the enhanced high-dimensional signal into a low-dimensional feature vector through principal component analysis and feature embedding techniques.

[0030] For example, the system processes a 0.1-second electric field time-domain signal containing 10,000 sampling points. First, an adaptive Kalman filter is applied, improving the signal-to-noise ratio by approximately 20 dB. Then, a three-layer 'db4' wavelet packet decomposition and reconstruction is performed, resulting in eight sub-band signals. After calculating and enhancing the local contrast of each sub-band, a 256-dimensional real vector, representing the electric field distribution feature vector, is generated using a dimensionality reduction algorithm.

[0031] Step S130: At the airborne edge computing node, the electric field distribution feature vector, the non-electric field modal data in the multi-physics sensing data stream, and the multi-angle image data stream are input into the cross-modal feature fusion network. Through feature-level fusion based on attention weights, a unified multi-dimensional fusion feature representation is generated.

[0032] Among them, the cross-modal feature fusion network is a deep neural network architecture used to integrate features of different types (modalities) of data into a unified representation; the modal feature encoder is a sub-network in the fusion network responsible for extracting features from the original data of a single modality; attention weight is a weight coefficient dynamically generated by the network, used to evaluate and assign the importance of different modal features in the fusion process; multi-dimensional fusion feature representation refers to the comprehensive feature tensor output by the fusion network that integrates information from all input modalities.

[0033] Specifically, the system can extract features from electric field distribution feature vectors, infrared temperature data, environmental humidity data, and multi-angle images using multiple independent modal feature encoders in a cross-modal feature fusion network (e.g., a one-dimensional convolutional network for electric field and temperature / humidity features, and a two-dimensional convolutional neural network for image features), obtaining the encoded features for each modality. Subsequently, the attention fusion layer in the network dynamically calculates a set of attention weights based on the data quality of each modal encoded feature (e.g., the signal-to-noise ratio of the infrared image and the completeness of the electric field data). Finally, the feature fusion layer multiplies each modal encoded feature by its corresponding attention weight, performs a weighted sum, concatenates it with the original encoded features, and compresses it through a fully connected layer to output a unified fused feature.

[0034] For example, the system inputs 256-dimensional electric field features, 64-dimensional temperature features, 2-dimensional humidity features, and five 512-dimensional visual feature vectors extracted from five images into the corresponding encoders. The network calculates the attention weights for each modality. After weighted fusion and concatenation compression, a 1024-dimensional real vector is generated as a unified multi-dimensional fusion feature representation.

[0035] Step S140: Transmit the multi-dimensional fusion feature representation to the ground computing node, use the lightweight target detection model deployed on the ground computing node to infer the multi-dimensional fusion feature representation, and output the identification result of the anomaly category on the insulator surface and the spatial coordinates of the anomaly location.

[0036] Among them, the lightweight object detection model is a deep learning model that has been structurally optimized to reduce parameters and computational load while maintaining high accuracy, and is used for object recognition and localization; the backbone network is the main part of the object detection model responsible for extracting multi-level features from the input data; the coordinate attention module is an attention mechanism component embedded in the model, which can enhance the model's ability to perceive the spatial location of features.

[0037] Specifically, the system can first reconstruct the received multi-dimensional fused feature representation into a feature map format suitable for convolutional network input, and then input it into a lightweight object detection model. The model performs preliminary feature extraction through its backbone network (which typically contains depthwise separable convolutions and inverse residual structures), then enhances key spatial features through a neck network integrating a coordinate attention module, and finally the detection head network predicts the position and size of the bounding box and the probability that the target within the box belongs to each anomaly category.

[0038] For example, the system reconstructs the 1024-dimensional fused feature representation into a 32-pixel × 32-pixel × 1-channel pseudo-image and inputs it into a lightweight object detection model based on an improved YOLO (You Only Look Once) architecture. After inference, the model outputs two predicted bounding boxes, one of which is classified as "damaged" with a confidence score of 0.92. The center coordinates of the bounding box in the image coordinate system are... The other box is categorized as "filth," with a confidence level of 0.87 and center coordinates of... These image coordinates can be converted into real-world three-dimensional spatial coordinates using calibration parameters.

[0039] Step S150: Based on the identification results and the spatial coordinates of the abnormal location, combined with the insulator operating context features, the structured insulator comprehensive status report is generated by evaluating the execution status classification of the model through dynamic threshold.

[0040] Among them, the insulator operating context features refer to the environmental, historical and asset information related to the insulator condition assessment, including but not limited to the years of operation, recent environmental temperature and humidity records, historical defect archives, etc.; the dynamic threshold assessment model is a machine learning model that can adaptively adjust the discrimination threshold according to the input context features; the structured insulator comprehensive condition report refers to the assessment document organized in a standardized format, which includes items such as anomaly type, location, severity level, and maintenance recommendations.

[0041] Specifically, the system can retrieve the full lifecycle operational data of the insulator being evaluated by accessing the asset database. This includes data such as being in operation for 5 years, having an average monthly humidity of 75%RH (relative humidity), and having one discharge history. These data are used as the insulator's operational context features. Subsequently, the context features, along with the identification results from step S140 (such as anomaly category and confidence level), are input into a dynamic threshold evaluation model (e.g., constructed based on a gradient boosting tree algorithm). This model outputs a dynamically calculated discrimination threshold for the current operating condition. The system compares the anomaly confidence level with this threshold and determines the state level according to preset mapping rules. Finally, the system automatically populates the report template, generating a structured report containing information for all entries.

[0042] For example, the system evaluates an insulator operating in a humid environment. The dynamic threshold assessment model outputs a dynamic discrimination threshold of 0.80 for "pollution" anomalies based on context. In the identification results, the confidence level for "pollution" is 0.87, which is higher than the threshold, so it is judged as "abnormal". Combining the extent to which its confidence level exceeds the threshold (8.75%) and historical records, the final status level is determined to be "moderately abnormal". The generated structured insulator comprehensive status report will include: anomaly type "pollution", location coordinates (X=105.3 m, Y=205.7 m, Z=15.2 m), level "moderately abnormal", and recommended maintenance measures "recommend cleaning within 3 months".

[0043] In another embodiment, such as Figure 2 This diagram illustrates the hardware and software functional modules and data flow of a multiphysics sensing and intelligent analysis system. Using a hierarchical rectangular frame format, the diagram clearly shows the logical relationships and data flow between the core processing modules throughout the entire process, from raw data acquisition to final status assessment and report generation.

[0044] Specifically, Figure 2 The leftmost multiphysics data acquisition module corresponds to the airborne sensing subsystem carried on the UAV platform in the above embodiments. This module is the data source of the system and is responsible for performing multi-dimensional data synchronous acquisition and spatiotemporal alignment. For example, in an inspection of an insulator string of a 500 kV transmission line, this module controls the electric field sensor to collect raw data for 1 second at a sampling rate of 1 MHz, the infrared thermal imager at a rate of 25 frames per second (fps), the temperature and humidity sensor at a frequency of 10 times per second (Hz), and the high-definition camera at 5 preset angles, and injects a unified spatiotemporal tag.

[0045] Figure 2 The electric field data processing module and the multimodal fusion analysis module, which follow closely behind, correspond to the software functions deployed in the airborne edge computing subsystem. The electric field data processing module is specifically responsible for performing the aforementioned signal preprocessing and feature encoding steps. For example, this module extracts and encodes a 256-dimensional electric field distribution feature vector from the 1 million raw electric field sampling points through adaptive Kalman filtering and a 3-layer 'db4' wavelet packet transform. The multimodal fusion analysis module is responsible for performing feature-level fusion based on attention weights. For example, this module combines the aforementioned 256-dimensional electric field features, the 64-dimensional non-electric field features encoded from infrared and temperature / humidity data, and the 512-dimensional visual features extracted from 5 images, through a cross-modal attention fusion network to generate a unified 1024-dimensional multi-dimensional fused feature representation. Figure 2The upper-level computer area, marked by the dashed box on the right, contains the improved lightweight YOLO target recognition module and the dynamic hierarchical state evaluation module. The improved lightweight YOLO target recognition module corresponds to the lightweight target detection model deployed on the ground computing server cluster, and is responsible for inference on the received fused feature representation. For example, this module reconstructs the 1024-dimensional fused features into... The system takes a pseudo-image as input, integrates a backbone network with a coordinate attention mechanism and a detection head, and outputs a prediction result within 30ms. For example, if it identifies an "umbrella skirt damage" anomaly with a confidence level of 0.94, it provides the bounding box coordinates. The dynamic grading status assessment module corresponds to the assessment software running on the operation and maintenance decision terminal, and is responsible for performing status grading. For example, this module calls the database to obtain the contextual features of the insulator, which has been in operation for 10 years and has a recent salt density of 0.18 mg / cm². Combined with the identified "damage" confidence level of 0.94, it calculates the current threshold of 0.85 through a dynamic threshold model, thereby determining the status level as "severe anomaly" and automatically generating a risk level code "3" and a maintenance suggestion code "B2" (meaning emergency handling within 24 hours). Figure 2 In the diagram, the modules are linearly connected by arrows, clearly indicating the direction of data flow: starting from the multiphysics data acquisition module, the data flows sequentially through the electric field data processing module and the multimodal fusion analysis module for edge-side processing, then is transmitted wirelessly to the improved lightweight YOLO target recognition module on the ground side for cloud-based intelligent recognition, and finally the results are sent to the dynamic hierarchical state assessment module to generate a decision report. This diagram intuitively reveals the "end-edge-cloud" collaborative processing architecture constructed in this invention, as well as the physical hardware support relationships and functional divisions of each core algorithm module.

[0046] In another embodiment, such as Figure 3 The diagram illustrates the process of collaboratively processing multi-physics measurement data to generate characteristic parameters of electric field distribution. Figure 3 The top-down chronological and logical sequence clearly reveals the specific implementation process and technical details of the core step in the above-mentioned related embodiments: "performing signal preprocessing and feature encoding of electric field measurement data in multi-physics sensing data stream at airborne edge computing nodes, and outputting electric field distribution feature vectors".

[0047] Specifically, the process begins with the collected multi-physics measurement data (electric field + temperature + humidity). This corresponds to the raw data stream acquired synchronously in the above embodiments. For example, when inspecting an insulator in a high-humidity coastal environment, the system collects a 0.2-second electric field time-domain waveform, a sequence of 20 infrared thermal images within the same time period, and 40 sets of temperature and humidity readings recorded simultaneously. These constitute the initial input of the process. Subsequently, the process enters the adaptive Kalman filtering denoising step. This step aims to suppress environmental electromagnetic noise and random interference in the original electric field signal. Specifically, the system updates the filtering parameters in real time based on the statistical characteristics of the signal and noise using the Kalman filtering algorithm. For example, this processing can enhance an original electric field signal with a signal-to-noise ratio of 18dB to 38dB, resulting in a smooth, denoised signal that highlights the true changes in the electric field. Next is the wavelet packet transform (multi-scale signal decomposition and reconstruction) step. This step performs fine time-frequency domain analysis on the denoised signal. Specifically, the system employs the "db4" wavelet basis function for three-level complete wavelet packet decomposition, resolving the signal into eight orthogonal sub-band components. For example, this transformation can separate the high-frequency oscillation component (50kHz to 80kHz) mainly caused by partial discharge, as well as the low-frequency background component (0Hz to 12.5kHz) reflecting the overall field strength distribution. The next step is local feature enhancement (strengthening areas of electric field distortion). This step amplifies the local abrupt change characteristics of each sub-band signal after wavelet packet decomposition. Specifically, the system calculates the energy or standard deviation of each sub-band signal within a sliding window and enhances the amplitude of high-fluctuation regions using a nonlinear gain function. For example, for a sub-band signal exhibiting a brief pulse at a specific time point, after enhancement, the amplitude of the pulse is increased to 2 to 3 times the background value, making it more significant in subsequent analysis. The core of the process is feature extraction (PCA+LLE dimensionality reduction, fusing temperature / humidity features). This step first concatenates the enhanced multi-scale signal (high-dimensional) with synchronously acquired temperature and humidity trend features. Specifically, the system first uses Principal Component Analysis (PCA) to linearly reduce the dimensionality of the stitched high-dimensional features, preserving the main variance information. Then, it employs Locally Linear Embedding (LLE) to maintain the local manifold structure of the data in the low-dimensional space, achieving nonlinear feature embedding. For example, the initial stitched feature vector might be 1000-dimensional, reduced to 100-dimensional by PCA, and then further compressed and embedded into a 16-dimensional dense vector using LLE. This 16-dimensional vector not only encodes the electric field distortion, gradient, and phase information but also incorporates the influence of temperature and humidity. The endpoint of the process is to obtain the electric field distribution characteristic parameters fused from the multiphysics model.This is the aforementioned electric field distribution feature vector, which is one of the key inputs to the subsequent cross-modal fusion network. For example, this feature parameter is a 16-dimensional real vector, and the values ​​of its different dimensions are related to physical properties such as the intensity of the electric field distortion on the insulator surface, the spatial distribution pattern, and the sensitivity to environmental humidity. Therefore, ... Figure 3 To illustrate intuitively, this invention transforms raw, noisy, and single-dimensional electric field measurement data into a low-dimensional, information-rich, and environmentally-integrated high-quality feature vector through a series of rigorous signal processing and feature engineering steps, laying a solid data foundation for a comprehensive and accurate assessment of insulator status.

[0048] In another embodiment, such as Figure 4 This diagram illustrates the process of multi-physics feature data processing and fusion. Presented in a sequential flowchart format, it clearly shows the internal processing logic and data transformation process of the core step described in the above embodiment: "At the airborne edge computing node, the electric field distribution feature vector, non-electric field modal data from the multi-physics sensing data stream, and multi-angle image data stream are input into the cross-modal feature fusion network. Through feature-level fusion based on attention weights, a unified multi-dimensional fused feature representation is generated."

[0049] Specifically, the process begins with acquiring multi-physics feature data (electric field + temperature + humidity + image). This corresponds to the state where the pre-processed modal data is ready. For example, for an insulator being tested, a 16-dimensional electric field distribution feature vector, a 64-dimensional temporal feature representing temperature changes, a 2-dimensional current humidity feature, and a 512-dimensional comprehensive visual feature vector extracted and pooled from five perspectives are already prepared. Subsequently, the process enters the feature encoding step (independent convolutional / fully connected encoding for each modal data). This step uses independent neural network sub-modules to map the original features of different structures and dimensions to a unified, higher-level feature space. Specifically, for the one-dimensional feature vector of the electric field, a one-dimensional convolutional network is used for encoding; for the scalar or low-dimensional temporal features of temperature and humidity, a fully connected network is used for encoding; and for the high-dimensional features of the image, a two-dimensional convolutional neural network is used for deep feature extraction and refinement. For example, after this step, the electric field feature is encoded as a 128-dimensional vector, the temperature and humidity feature as a 64-dimensional vector, and the image feature as a 512-dimensional vector, all three possessing the same level of abstraction and comparability. Next is the attention fusion (dynamically allocating modal weights based on data reliability) step. This step is one of the core innovations of the fusion network, aiming to adaptively adjust the contribution of each modal input data to the final fused feature based on the real-time quality assessment results. Specifically, the system analyzes the signal-to-noise ratio, integrity, and other indicators of each modal encoded feature through a small evaluation sub-network, calculates its respective reliability score, and normalizes it into a set of attention weights that sum to 1 using the Softmax function. For example, in this instance, the electric field signal is clear, the image is slightly blurry, and the temperature and humidity data are stable; the calculated dynamic weights might be allocated as follows: electric field 0.40, visual 0.35, and temperature and humidity 0.25. The next step is feature fusion (element-wise weighted summation + channel-level concatenation). "Element-wise weighting," also known as element-level weighting, refers to applying weights independently to each element of a tensor or matrix, commonly seen in activation functions or weighted calculations in deep learning. This step first performs element-wise weighted summation with the corresponding modal's encoded features to obtain a preliminary fused feature. Then, this preliminary fused feature is concatenated with the original encoded features of each modality along the channel dimension to form a more information-rich combined feature. For example, the 256-dimensional preliminary fused feature obtained from the weighted summation is concatenated with a 128-dimensional electric field encoded feature, a 64-dimensional temperature and humidity encoded feature, and a 512-dimensional visual encoded feature, ultimately resulting in a 960-dimensional extended feature vector. The end result of this process is to obtain a multimodal fused feature vector, enabling the discovery of potential correlations across multiple physics fields.The extended feature vector obtained after the aforementioned steps will undergo nonlinear transformation and compression through subsequent fully connected layers, ultimately outputting a fixed-dimensional, unified multi-dimensional fused feature representation. For example, the 960-dimensional extended features are compressed and refined into a 1024-dimensional fused feature vector through a two-layer fully connected network. This vector deeply integrates electric field, thermal, humidity, and visual information, and its internal representation can effectively mine and characterize the potential correlations between these physical fields and the insulator state. For example, "electric field distortion accompanied by local temperature rise in a high humidity environment" may correspond to a "pollution discharge" mode.

[0050] therefore, Figure 4 The key operational process of the cross-modal feature fusion network in this invention is explained intuitively and in detail. This process, through independent encoding, attention weighting, and hierarchical fusion, not only achieves effective integration of multi-source heterogeneous data, but more importantly, through data-driven weight allocation, it adaptively focuses on more reliable modal information, thereby generating a high-quality fusion feature that can profoundly reveal the potential correlations of multiple physics fields, providing a crucial information foundation for subsequent accurate identification.

[0051] In another embodiment, such as Figure 5 This diagram illustrates the complete training and deployment process of the improved lightweight YOLO object detection model. Presented as a structured flowchart, it systematically explains the entire chain of steps, from preparing fused feature data and optimizing model training to final deployment and application. It clearly reveals the model construction methods and quality control processes underlying the "inference using a lightweight object detection model deployed on ground computing nodes" step in the above embodiment.

[0052] Specifically, the process begins at the start node and immediately enters the data preparation stage: multi-physics fusion feature vectors (electric field + temperature + humidity + image). This corresponds to using the high-quality fusion features generated in the preceding steps as the basic raw material for model training. For example, the system accumulates a dataset containing 1000 insulator samples, each sample corresponding to a 1024-dimensional multi-physics fusion feature vector and its labeled anomaly bounding boxes and class labels. Subsequently, the process performs a dataset partitioning operation, scientifically dividing it into training, validation, and test sets. This is a crucial step in ensuring the model's generalization ability. Specifically, the system randomly partitions the dataset according to a 70%:15%:15% ratio. For example, 1000 samples are divided into 700 for training, 150 for validation, and 150 for final testing. The core component is the iterative training of the improved lightweight YOLO network (coordinate attention + adaptive anchor boxes). This step is further subdivided into: Forward propagation (including depthwise separable convolution + coordinate attention inference): The model receives training set data and performs inference computation through its lightweight backbone network (integrating depthwise separable convolution) and coordinate attention module.

[0053] Calculate the loss (classification loss + localization loss + attention regularization loss): The system calculates the difference between the model's prediction and the actual annotation. The loss function integrates the class judgment error, the bounding box position error, and the regularization constraint on the attention weights to simultaneously optimize recognition accuracy and model complexity.

[0054] Backpropagation (transfer learning initialization + mixed-precision training): Gradients are backpropagated using the loss value to update network parameters. To accelerate convergence and improve stability, training uses weights pre-trained on a large public dataset for initialization and applies mixed-precision training techniques.

[0055] The training process includes a strict stopping and evaluation mechanism. The system continuously checks whether the training stopping conditions have been met (e.g., reaching the maximum number of training epochs or the validation set loss no longer decreasing). If not met (N), it continues iterating; if met (Y), the model performance is evaluated using the validation set, and it is further determined whether the model performance meets the requirements (e.g., mAP@0.5 is higher than 0.90). If not met (N), the hyperparameters are adjusted and retraining is performed; if met (Y), the model is finally evaluated using the test set to obtain objective performance metrics (e.g., precision, recall) on unknown data.

[0056] The ultimate goal of the process is to deploy a lightweight insulator anomaly identification model (adapted to edge computing) to identify insulator anomalies and their locations from real-time multiphysics data. For example, the final model that meets performance requirements is converted to TensorRT or ONNX (Open Neural Network Exchange) format, optimized, and then deployed on a ground-based computing server cluster. When new inspection data generates a fused feature representation and is input into the model, the system can output, within milliseconds, information such as "Detected 'mandrel crack,' confidence level 0.96, location coordinates...". "A reliable result."

[0057] therefore, Figure 5 This invention fully demonstrates the closed-loop engineering practices employed to ensure recognition accuracy and reliability, from data preparation, model training, verification and evaluation to production deployment. This rigorous process is a crucial technical guarantee for generating high-quality recognition results and spatial coordinates of anomaly locations, ultimately supporting accurate state classification.

[0058] Therefore, according to the above implementation method, the system acquires the temperature contrast-enhanced image, tilt angle, and lateral offset distance of the composite insulator through the infrared image acquisition module, which is used to accurately extract the key geometric and state parameters of the target from the infrared image; based on these parameters, the intelligent analysis module reverse-engineers the boundary contour of the umbrella skirt shading area and extracts irregular shape features to calculate the tilt width value, which is used to quantify the degree of lateral offset of the core rod; based on the shape influence factor determined by the tilt width value, the system analyzes its correspondence with the area of ​​the shading area and obtains the trend characteristics of the fluctuation curve through regression fitting, which is used to establish a dynamic mapping model between the tilt state and the shading area; based on the trend characteristics of the fluctuation curve, the system generates the target shooting position and angle parameters of the flying inspection vehicle through simulated shooting data analysis, which is used to determine the optimal shooting strategy; based on the target shooting position and angle parameters, the system updates the inspection path and controls the flying inspection vehicle to perform infrared temperature measurement tasks, which is used to achieve accurate data acquisition.

[0059] Specifically, in the technical solution of this embodiment, addressing the problem that existing methods described in the background art are unable to adaptively adjust the shooting strategy according to the dynamically changing tilt width, the occlusion contour is inverted based on the tilt angle and lateral offset distance, and the precise tilt width value is calculated. This achieves accurate quantification and characterization of the umbrella skirt occlusion shape, thus providing key input for the subsequent shooting strategy formulation and solving the defect of existing technologies that cannot dynamically adjust the shooting pose due to a lack of precise perception of the occlusion area. Addressing the problem of insufficient decision-making basis for shooting position and angle under complex working conditions, a quantitative model for predicting occlusion under different tilt states is constructed by establishing a mapping relationship between shape influence factors and occlusion area and fitting its fluctuation curve trend. This solves the drawbacks of traditional fixed path or preset angle methods having poor adaptability and failing to guarantee data integrity in highly dynamic inspection environments. Addressing the problem of the shooting strategy being out of sync with real-time working conditions, the target shooting position and angle parameters are dynamically generated based on the fluctuation curve trend characteristic analysis results, and the flight inspection path is updated accordingly. This establishes a closed-loop adjustment mechanism of "perception-analysis-decision-execution," solving the shortcomings of existing technologies where shooting strategies are rigid and unable to respond to real-time changes in target status. Therefore, the technical solution of this embodiment solves the technical problem of insufficient accuracy in assessing the overall operating status of insulators in existing drone-based insulator inspection technologies, and improves the accuracy and adaptability of infrared temperature measurement front-end data acquisition.

[0060] In some embodiments, multi-dimensional data synchronous acquisition and spatiotemporal alignment are performed to obtain multi-physics sensing data streams and multi-angle image data streams of the insulator, including: By deploying a multi-physics field sensing array on a drone platform, the original electric field signal, infrared thermal radiation signal, and microenvironment temperature and humidity signal of the insulator are collected concurrently and aggregated to generate original multimodal sensing data.

[0061] Concurrent acquisition refers to the use of a unified hardware trigger signal or software instruction to control the various sensors in the multiphysics field sensing array to start data acquisition at the same or very close time points, so as to ensure the temporal alignment of different physical quantity measurements; raw multimodal sensing data refers to the set of data temporarily stored in chronological order of the raw, unprocessed measurement values ​​collected by each sensor.

[0062] Specifically, the system can send a hardware trigger pulse to all sensors simultaneously when the UAV flight control system reaches a preset data acquisition point. Each sensor then begins acquiring data at its own set sampling rate and sends time-stamped data packets to the onboard data aggregation unit in real time via a high-speed bus for caching and initial packaging. For example, when the UAV hovers 10 meters directly in front of an insulator, the flight control system issues a trigger signal. The electric field sensor begins recording the electric field strength at a sampling rate of 1 MHz, the infrared thermal imager begins outputting temperature distribution images at a rate of 25 fps, and the temperature and humidity sensor begins recording data at a frequency of 10 times per second (Hz). Within a 2-second acquisition window, these three types of data are aggregated in real time to form a raw multimodal sensor data packet containing 2 million electric field sampling points, 50 frames of infrared images, and 20 sets of temperature and humidity readings.

[0063] Using a high-definition vision sensor deployed on the same platform, and based on preset track points and observation attitudes, a multi-view image frame sequence of the insulator is acquired.

[0064] Among them, the preset flight path points and observation attitudes refer to the three-dimensional spatial coordinates of the UAV and gimbal camera pre-set during the inspection mission planning stage. And camera pointing angle (yaw angle, pitch angle, roll angle); a multi-view image frame sequence refers to a series of images taken of the same target from different spatial positions and angles in a certain order.

[0065] Specifically, the system can load a preset inspection route file, control the drone to sequentially fly to various waypoints, and after hovering stably at each point, adjust the gimbal angle according to the preset observation attitude parameters for that point, and then control the high-definition vision sensor to take pictures. The captured images are stored together with the corresponding waypoint numbers and attitude parameters. For example, for a tension insulator string, the system presets 5 waypoints, located directly in front of the insulator string, 45 degrees to the left front, 45 degrees to the right front, directly above (looking down), and directly below (looking up). At each point, the gimbal is adjusted to position the insulator string in the center of the frame. The system sequentially flies to these 5 points and captures a 4K resolution (3840×2160 pixels) color image at each point, thus obtaining a multi-view image frame sequence containing 5 perspectives.

[0066] For each data unit in the original multimodal sensing data and multi-view image frame sequence, inject a unified time stamp and a coordinate label based on spatial positioning.

[0067] Among them, the unified time stamp refers to the absolute time information originating from the same high-precision clock source (such as the airborne GPS timing module); the coordinate label based on spatial positioning refers to the three-dimensional position (longitude, latitude, altitude) and carrier attitude information calculated in real time by the UAV integrated navigation system (integrating GNSS and IMU) in a specified coordinate system (such as the WGS-84 coordinate system).

[0068] Specifically, the system can use an onboard data acquisition computer to immediately read the current precise time (usually accurate to the microsecond level) and spatial pose information from a high-precision clock and integrated navigation system upon receiving each sensor data packet or each image frame, and attach this information as metadata tags to the data unit. For example, for an image frame taken at a certain waypoint, the system injects a timestamp of "2023-10-26 14:30:05.123456 UTC (Coordinated Universal Time)" and coordinate tags of "Longitude: 118.123456 degrees, Latitude: 32.654321 degrees, Altitude: 85.5 meters, UAV yaw angle: 120.5 degrees, Pitch angle: 120.5 degrees, Latitude ... Pitch angle: 120.5 degrees, Pitch angle: 120.5 degrees, Pitch angle: 120.5 degrees, Pitch angle: 120.5 degrees, Pitch angle: 120.5 degrees, Pitch angle: 120.5 degrees, Pitch angle: 120.5 degrees, Pitch angle: 120.5 degrees, Pitch angle: 1 "Temperature, roll angle: 1.1 degrees". For a segment of electric field data collected at the same time, the start and end times are also calibrated using the same clock source.

[0069] Using the injected timestamps and coordinate labels as indexes, the original multimodal sensing data and multi-view image frame sequences are spatiotemporally registered and aligned, outputting a spatiotemporally synchronized multiphysics sensing data stream and a multi-angle image data stream.

[0070] Spatiotemporal registration refers to the process of mapping data collected from different sensors at different times to a unified spatiotemporal reference framework using time and space labels; sequence alignment refers to processing data streams with different sampling rates so that their data points can correspond one-to-one on the timeline or establish a clear interpolation correspondence.

[0071] Specifically, the system can first convert the timestamps of all data to the same time base using the data processing unit. Then, using the capture time of the multi-view image frame sequence as the reference time, it performs coordinate transformation on the original multimodal sensing data using the position and attitude information in the coordinate labels, mapping it to a local coordinate system with the insulator as the origin. For sensing data with a sampling rate higher than the image frame rate (such as electric field data), interpolation is performed before and after the image capture time to generate sensing data samples that strictly correspond to the image capture time, thereby completing sequence alignment. For example, the system uses five precise times (T1 to T5) of images captured from five track points as a reference. For a temperature and humidity reading acquired at time T3.1 (slightly later than T3), the system performs spatial interpolation based on the coordinate labels at times T3 and T4 to calculate the sensor's position relative to the insulator at time T3.1, and assigns this reading to time T3. For electric field data sampled at 1000 Hz, the system takes several points before and after time T3 and performs sinc interpolation to accurately reconstruct the electric field signal value at time T3. After the above processing, five spatiotemporally synchronized data packets are finally output. Each packet contains: one image taken at time Ti, one set of electric field / temperature and humidity data interpolated and aligned at time Ti, and one frame of infrared thermal image that is closest to time Ti in time.

[0072] Therefore, according to the above implementation method, the system can ensure that heterogeneous data acquired from different sensors at different moments are highly consistent in time and space, providing an accurate and aligned data foundation for subsequent multimodal feature fusion and collaborative analysis, and fundamentally avoiding feature matching errors and state misjudgments caused by data asynchrony.

[0073] In some embodiments, the electric field measurement data in the multiphysics sensing data stream is preprocessed and feature-encoded at an airborne edge computing node to output an electric field distribution feature vector, including: Adaptive filtering is performed on the input electric field measurement data stream to suppress environmental noise and random interference, and the noise-reduced electric field time-domain signal is output.

[0074] Adaptive filtering is a signal processing technique that can automatically adjust filter parameters according to the characteristics of the input signal, aiming to optimally separate useful signals from noise.

[0075] Specifically, the system can be implemented by running an adaptive Kalman filter algorithm on an airborne edge computing node. This algorithm dynamically updates the Kalman gain by estimating the statistical properties (covariance matrix) of the electric field signal and noise in real time, thereby effectively suppressing random noise and impulse interference during the filtering process while preserving the original electric field variation characteristics that characterize the insulator state to the greatest extent possible. For example, the system processes a 0.1-second segment of raw electric field measurement data (10,000 sampling points) with a sampling frequency of 100 kHz. The signal-to-noise ratio (SNR) of the raw signal is approximately 15 dB. After adaptive Kalman filtering, the random noise of the output signal is significantly suppressed, and the SNR is improved to approximately 35 dB, resulting in a smooth, denoised electric field time-domain signal that reflects the basic trend of the electric field.

[0076] A multi-scale time-frequency transform is performed on the denoised electric field time-domain signal to extract the multi-scale frequency domain components.

[0077] Among them, multiscale time-frequency transform is a mathematical tool that can perform localized analysis of signals in both the time and frequency domains, and is used to reveal the characteristics of signals at different time scales and frequency components; multiscale frequency domain components refer to the set of sub-band signals corresponding to different frequency ranges or scales obtained after transforming the signal.

[0078] Specifically, the system can achieve this by performing wavelet packet transform on the denoised electric field time-domain signal. Based on discrete wavelet transform, wavelet packet transform further decomposes the high-frequency subbands, thus providing more refined time-frequency analysis covering the entire frequency band. The system selects specific wavelet basis functions (such as the 'db4' wavelet) and decomposition levels to decompose the signal into a series of wavelet packet coefficients with different center frequencies and bandwidths; these coefficients constitute the multi-scale frequency domain components. For example, the system performs 3-level 'db4' wavelet packet decomposition on the aforementioned 0.1-second denoised time-domain signal. After decomposition, the original signal is resolved into 8 (… The signal consists of multi-scale frequency domain components, each corresponding to a specific sub-band. For example, the first component (low frequency) mainly contains components from 0Hz to 12.5kHz, while the eighth component (high frequency) contains components from 87.5kHz to 100kHz, completely covering the signal's frequency spectrum.

[0079] Local feature response enhancement is performed on multi-scale frequency domain components to strengthen the feature response of distorted regions in each signal component.

[0080] Local feature response enhancement is a signal processing operation that aims to make potential, weak abnormal distortion features in a signal more prominent by calculating and amplifying the contrast or intensity of changes in local regions of the signal, thus facilitating subsequent detection.

[0081] Specifically, the system quantifies the local fluctuation intensity of a signal by calculating the local energy or local standard deviation of each multi-scale frequency domain component (i.e., the wavelet packet coefficient sequence). Then, the original signal is enhanced using a nonlinear mapping function (e.g., multiplied by a gain coefficient positively correlated with the local fluctuation intensity), resulting in a relatively amplified amplitude in regions of intense fluctuation (potentially corresponding to electric field distortion points), while the amplitude changes in stable regions are smaller. For example, for one of the eight wavelet packet coefficient sequences mentioned above, the system calculates the standard deviation of the coefficients within each window using a sliding window of 50 sampling points. Assuming that near a certain time point, the standard deviation within the window is 0.15 (relatively high), while the average standard deviation of the sequence is 0.05, the system assigns an enhancement factor of 1.5 to that point, multiplying the wavelet packet coefficient values ​​near that point by 1.5, thereby strengthening the characteristic response of that distorted region in the frequency domain.

[0082] Feature embedding and dimensionality reduction are performed on the enhanced feature response to encode a low-dimensional feature vector that integrates the characteristics of electric field distortion, field strength gradient and phase shift, serving as the electric field distribution feature vector.

[0083] Feature embedding refers to the process of mapping a high-dimensional, potentially sparse signal representation that has undergone enhancement processing to a new, denser, and more expressive vector space through linear or nonlinear transformations; dimensionality reduction refers to reducing the dimensionality of feature data through mathematical methods while preserving as much of the main information and structure in the original data as possible, in order to reduce subsequent computational complexity and avoid the "curse of dimensionality".

[0084] Specifically, the system first concatenates all enhanced multi-scale frequency domain components (wavelet packet coefficients) sequentially into a high-dimensional feature. Then, principal component analysis (PCA) is used to reduce the dimensionality of this high-dimensional feature, extracting the most dominant feature directions (principal components). Next, methods such as local linear embedding are used to preserve the local geometric structure of the data in the low-dimensional space, achieving feature embedding. The resulting low-dimensional vector contains a compressed representation extracted from the original signal that comprehensively reflects electric field distortion (abrupt changes in energy within a specific frequency band), field strength gradient (rate of change of the signal envelope), and phase shift (instantaneous phase information extracted through Hilbert transform). For example, the system concatenates eight enhanced wavelet packet coefficient sequences (each sequence approximately 1250 points long) into a 10000-dimensional original feature vector. After PCA, the first 50 principal components are retained, reducing the dimensionality to 50 dimensions. Further local linear embedding generates a 16-dimensional real vector. This 16-dimensional vector is the electric field distribution characteristic vector. The magnitude of its values ​​in different dimensions is associated with specific electric field distortion modes, gradient change intensity, or phase jump characteristics.

[0085] Therefore, according to the above implementation method, the system can perform efficient and automated deep processing and feature extraction on raw, noisy electric field measurement data streams in a resource-constrained airborne edge computing environment. This process, from noise suppression, multi-scale analysis, and local enhancement to intelligent compression coding, gradually extracts low-dimensional, dense feature vectors that best characterize the anomalies in the electric field state of the insulators, providing high-quality, high-information electric field feature inputs for subsequent fusion with multimodal data such as temperature, humidity, and images.

[0086] In some embodiments, at an airborne edge computing node, the electric field distribution feature vector, non-electric field modal data from the multi-physics sensing data stream, and multi-angle image data stream are input to a cross-modal feature fusion network. Through attention-weight-based feature-level fusion, a unified multi-dimensional fused feature representation is generated, including: The electric field distribution feature vector, non-electric field modal data, and multi-angle image data stream are respectively input into independent modal feature encoders to extract electric field coding features, non-electric field coding features, and visual coding features.

[0087] Among them, the modal feature encoder is a neural network submodule in the cross-modal feature fusion network specifically designed to process single-type (modal) input data. Its function is to map the original or pre-processed modal data to a high-dimensional, abstract feature space. In this embodiment, non-electric field modal data specifically refers to infrared temperature data and environmental humidity data in the multi-physics sensing data stream, excluding electric field measurement data. Visual coding features refer to high-dimensional feature representations extracted from multi-angle image data streams that characterize the visual appearance and texture of insulators.

[0088] Specifically, the system can be implemented by deploying three structurally independent modal feature encoders within a cross-modal feature fusion network. For the electric field distribution feature vector (e.g., a 16-dimensional vector), a one-dimensional convolutional network with two fully connected layers is used for encoding, boosting it to a fixed high dimension. For non-electric field modal data (temperature and humidity scalar sequences), a shallow fully connected network is used for encoding. For multi-angle image data streams, a lightweight two-dimensional convolutional neural network (e.g., a variant of MobileNetV3) is used to extract features from each image, and then the features from all images are averaged or max-pooled to obtain a comprehensive visual feature. For example, the system inputs a 16-dimensional electric field distribution feature vector into a one-dimensional convolutional encoder, outputting a 128-dimensional electric field encoded feature. Inputting a sequence containing 20 temperature readings and 20 humidity readings into a fully connected encoder outputs a 64-dimensional non-electric field encoded feature. Five preprocessed insulator images (adjusted to 224×224 pixels) are input into the MobileNetV3 encoder. A 512-dimensional feature is extracted from each image. The average of the five 512-dimensional features is then calculated to obtain a 512-dimensional visual coding feature.

[0089] Based on the signal quality and data integrity corresponding to electric field coding features, non-electric field coding features and visual coding features, the reliability score of each modal feature is calculated, and the modal attention weight is dynamically allocated according to the reliability score.

[0090] Among them, signal quality and data integrity are a set of indicators used to quantitatively evaluate the credibility and effectiveness of each modal feature; reliability score is a scalar value used to comprehensively represent the reliability of the corresponding modal feature; modal attention weight is a probability distribution vector generated by the network with the same dimension as the number of modalities, and the value of each element represents the degree to which the corresponding modal feature should be valued in the fusion process, and the sum of all weights is 1.

[0091] Specifically, the system can compute reliability scores using a small, learnable evaluation subnetwork within the fusion network. This subnetwork takes the encoded features of each modality as input and outputs a score. The score can be based on various pre-defined metrics; for example, for electric field and temperature / humidity modalities, the signal-to-noise ratio can be evaluated; for image modalities, sharpness (e.g., by calculating the Laplacian variance of the image) and data integrity (e.g., the percentage of effective pixels) can be evaluated. The three calculated reliability scores are then combined... Normalization is performed using a Softmax function, thereby dynamically allocating the corresponding modal attention weights. For example, suppose the evaluation subnetwork calculates a reliability score for the electric field characteristics. The score is 0.9 (high signal-to-noise ratio), and the non-electric field characteristic score is 0.9. The score was 0.7 (data is complete but slightly fluctuating), for visual feature scoring. The value is 0.8 (the image is clear but slightly hazy). Calculated using the Softmax function: This yields the dynamically allocated modal attention weight vector. .

[0092] Using the assigned modal attention weights, element-wise weighted summation is performed on the electric field encoded features, non-electric field encoded features, and visual encoded features respectively to obtain preliminary fused features.

[0093] Among them, element-wise weighted summation refers to the operation of multiplying multiple feature vectors of the same dimension by their corresponding scalar weights, and then summing the elements at corresponding positions to obtain a new feature vector of the same dimension.

[0094] Specifically, the system first needs to project the electric field encoded features, non-electric field encoded features, and visual encoded features onto the same dimension (e.g., 256 dimensions) through a fully connected layer. Then, the assigned weights are... The electric field characteristics after projection Non-electric field characteristics Visual features Multiply, and finally sum the three weighted eigenvectors: The result This represents the initial fusion features. For example, the three features after projection. All are 256-dimensional vectors. The weights are... Preliminary fusion characteristics of system computing The value of the first element is: This process is repeated for all 256 elements to generate a preliminary 256-dimensional fused feature vector.

[0095] The initial fused features are concatenated with the original encoded features of each modality using tensors, and then the features are transformed and compressed through a fully connected layer to output a unified multi-dimensional fused feature representation.

[0096] Tensor concatenation refers to the operation of connecting multiple feature vectors or feature maps along a specific dimension (usually the channel dimension) to form a combined feature with a larger dimension.

[0097] Specifically, the system can initially fuse 256-dimensional features. The original 128-dimensional electric field encoded features, the original 64-dimensional non-electric field encoded features, and the original 512-dimensional visual encoded features are concatenated along the channel dimension to obtain an extended feature vector with a total dimension of 256 + 128 + 64 + 512 = 960 dimensions. This 960-dimensional vector is then input into one or more fully connected layers for nonlinear transformation and dimensionality compression, ultimately outputting a dense feature vector of a fixed dimension (e.g., 1024 dimensions), i.e., a unified multi-dimensional fused feature representation. For example, the system inputs the 960-dimensional concatenated features into a two-layer fully connected network: the first layer has 512 neurons using the ReLU (Rectified Linear Unit) activation function; the second layer has 1024 neurons. Finally, the network outputs a 1024-dimensional vector. This vector integrates all information from the electric field, temperature, humidity, and image, and undergoes reliability-based weighted fusion and further deep integration, becoming a high-quality input for subsequent object recognition models.

[0098] Therefore, according to the above implementation method, the system can intelligently evaluate and utilize the reliability of data from different sources, achieving adaptive and weighted deep fusion at the feature level. This method avoids the information redundancy or noise amplification problems that may result from simple splicing, ensuring that even when the quality of some modal data is poor, the fusion process tends to trust more reliable modalities, thereby generating a more robust and complete unified multi-dimensional fusion feature representation, laying a solid data foundation for accurate identification of insulator anomalies.

[0099] In some embodiments, the multi-dimensional fused feature representation is transmitted to a ground computing node, and a lightweight target detection model deployed on the ground computing node is used to infer the multi-dimensional fused feature representation, outputting the identification result of the anomaly category on the insulator surface and the spatial coordinates of the anomaly location, including: The multi-dimensional fused feature representation is input into the backbone network of the lightweight object detection model, and forward propagation and feature mapping are performed through the deep separable convolutional layers and inverse residual structures in the backbone network.

[0100] Among them, the backbone network is the core part of the object detection model responsible for extracting multi-level and abstract features from the input data; the depthwise separable convolutional layer is a convolutional neural network layer that decomposes the standard convolution into two independent steps: depthwise convolution and pointwise convolution, aiming to reduce the number of model parameters and computational cost; the inverse residual structure is a block structure that first increases the number of channels through pointwise convolution, then performs depthwise convolution, and finally reduces the number of channels through pointwise convolution. It is often used in lightweight networks and can reduce computational overhead while maintaining or even improving feature representation capabilities.

[0101] Specifically, the system first reshapes the received multi-dimensional fused feature representation (e.g., a 1024-dimensional vector) into a two-dimensional feature map format suitable for convolutional network input (e.g., 32 pixels high × 32 pixels wide × 1 channel). Then, this feature map is input into the backbone network of a lightweight object detection model. This backbone network typically consists of multiple stacked basic building blocks containing depthwise separable convolutional layers and inverse residual structures, which downsample and abstract the input feature map layer by layer, outputting one or more deep feature maps with rich semantic information. For example, the system reshapes the 1024-dimensional fused feature representation into a 32×32×1 pseudo-image and inputs it into a lightweight network with MobileNetV2 as its backbone. This backbone network contains multiple inverse residual structure modules. After forward propagation, the original input is progressively mapped into a deep feature map of 8 pixels high × 8 pixels wide × 256 channels, which contains high-level semantic information extracted from the fused features for object detection.

[0102] The feature map output from the backbone network is input into the coordinate attention module of the lightweight object detection model to recalibrate the spatial dimension weights of the feature map.

[0103] Among them, the coordinate attention module is an attention mechanism component embedded in the neural network. It can aggregate information along the height and width of the feature map and generate a pair of direction-aware attention maps, thereby enhancing the network's ability to capture target spatial location information. Spatial dimension weight recalibration refers to using the attention map to assign a weight coefficient to each spatial location (i.e., each pixel) in the feature map, and readjusting the strength of the feature response at each location in the feature map through element-wise multiplication.

[0104] Specifically, the system will output the feature map from the backbone network (e.g., a size of...). The input coordinate attention module first performs global average pooling on the feature map along both the height and width directions, obtaining feature vectors in both directions. Then, these two vectors are fed into a sub-network with shared weights for processing, and a height attention weight map (size 1) is generated by applying a sigmoid activation function to each. ) and width attention weight map (size is Finally, the original feature map is element-wise multiplied with these two attention maps to recalibrate the weights of the feature map in the height and width directions, highlighting features relevant to the target location and suppressing irrelevant background features. For example, for an 8×8×256 feature map, the coordinate attention module outputs a height attention map (8×1×1) and a width attention map (1×8×1). Assuming the weight in the 3rd row of the height map is 1.5 and the weight in the 5th column of the width map is 1.2, then the feature map located in the 3rd row of the height map will have a weight of 1.5. The eigenvalues ​​of all 256 channels of the location will be multiplied by This process enhances the feature map. After this operation, a feature map with spatial dimension weights recalibrated is output, still retaining a size of 8×8×256.

[0105] The weighted and recalibrated feature map is input into the detection head of the lightweight target detection model.

[0106] The detection head is the part attached to the backbone network and neck network in the target detection model. It is responsible for receiving the feature maps that have been extracted and enhanced, and performing specific classification (determining the target category) and regression (predicting the target bounding box location) tasks.

[0107] Specifically, the system inputs the spatially enhanced feature map, processed by the backbone network and coordinate attention module, into the detection head of the lightweight object detection model. The detection head typically consists of a few convolutional or fully connected layers. Its input is the feature map, and its output is a dense set of predictions—a series of predicted values ​​corresponding to each pre-defined anchor point on the feature map. For example, the system inputs the aforementioned 8×8×256 enhanced feature map into the detection head. This detection head can be a simple 3×3 convolutional layer followed by two parallel 1×1 convolutional layers, used for classification and regression, respectively. The spatial grid of the input feature map is 8×8. Assuming each grid position has three pre-defined anchor boxes of different scales, the detection head will output a classification score vector (e.g., the probability of containing four categories: "normal," "damaged," "dirty," and "discharging") and a bounding box offset vector (containing four values ​​used to adjust the position and size of the pre-defined anchor boxes) for each of these 8×8×3=192 anchor boxes.

[0108] The adaptive anchor box generation layer and the classification and regression layer of the detection head are computed in parallel, and the output includes the anomaly category label, category confidence and target bounding box coordinates. The target bounding box coordinates are the spatial coordinates of the anomaly location.

[0109] The adaptive anchor box generation layer is part of the detection head. Its function is to automatically generate a set of best-matching anchor box sizes based on the actual size and aspect ratio distribution of the insulator targets in the training dataset, using a clustering algorithm (such as K-means, an unsupervised clustering analysis algorithm). This replaces manually preset fixed-size anchor boxes, thereby improving the model's adaptability to targets of different sizes. The classification and regression layers are two parallel output layers in the detection head. The classification layer is responsible for predicting the probability that the target in each anchor box belongs to each category, while the regression layer is responsible for predicting the fine-tuning (offset and scaling) that each anchor box needs to make to more closely surround the real target.

[0110] Specifically, during the model training phase, the adaptive anchor box generation layer performs cluster analysis based on the true dimensions of all insulator bounding boxes in the labeled dataset, generating K groups (e.g., 3 groups) of the most representative anchor box widths and heights. During the inference phase, the system simultaneously inputs the weighted feature maps into the classification and regression layers of the detection head. The classification layer outputs a class probability distribution for each preset anchor box at each anchor point location, while the regression layer outputs four values ​​for each anchor box (typically representing center point coordinate offset and width / height scaling). The system then applies post-processing algorithms such as Non-Maximum Suppression (NMS) to filter out the final predicted boxes with the highest confidence and most accurate locations from a large number of overlapping predicted boxes. The class of these final predicted boxes (taking the class with the highest probability) is the anomaly class label, and its corresponding probability is the class confidence. The coordinates of these final predicted boxes in the image coordinate system (typically represented as...) after decoding are... The coordinates of the target bounding box are shown below. These coordinates can be converted into real-world spatial coordinates through camera calibration and coordinate inversion. For example, for a given predicted bounding box, the classification layer outputs a probability of... The values ​​correspond to "normal", "damaged", "dirty", and "discharge" respectively. Therefore, the anomaly category label is "damaged", with a category confidence level of 0.92. The regression layer output offset is... Combining the preset anchor box corresponding to the anchor point of the predicted bounding box (assumed to be [20 pixels, 15 pixels]), the decoded target bounding box coordinates are: center point The bounding box is 24 pixels wide and 16.5 pixels high. Its position in the image represents the spatial coordinates of the "damage" anomaly.

[0111] Therefore, according to the above implementation method, the system can utilize the more powerful computing resources of ground computing nodes to run complex, lightweight target detection models. This model abstracts features through an efficient backbone network, accurately focuses on abnormal regions using a coordinate attention mechanism, and outputs high-precision identification and localization results through an adaptive detection head. The entire inference process achieves automated and accurate detection and localization of various abnormal states on the insulator surface, providing direct input for subsequent state assessment.

[0112] In some embodiments, based on the identification results and the spatial coordinates of the abnormal location, combined with the insulator operating context features, a structured insulator comprehensive status report is generated by evaluating the state classification of the dynamic threshold assessment model, including: Load the insulator's full lifecycle operation data and environmental sensing data as insulator operation context features.

[0113] Among them, the full life cycle operation data refers to the historical records and statistical information related to the insulator assets being evaluated since they were put into operation, including but not limited to the years of operation, inspection records, historical defects and maintenance files, load current curves, etc.; environmental sensing data refers to long-term or real-time data related to the operating environment of the insulators, such as monthly average humidity, monthly pollution level, ambient temperature, etc., collected from fixed monitoring points deployed on transmission lines or substations, or from this drone inspection.

[0114] Specifically, the system can query and retrieve the insulator's full lifecycle operational data by calling the API (Application Programming Interface) of the Asset Management System (AMS) based on the insulator's unique asset number (such as tower number and phase). Simultaneously, the system retrieves recent (e.g., the past month) environmental sensor data statistics for the area where the insulator is located from the environmental monitoring database. These two types of data together constitute the background information needed to assess the current condition of the insulator, namely, the insulator's operational context characteristics. For example, the system loads data from an insulator operating in a coastal area. Full lifecycle operational data shows that the insulator has been in operation for 8 years, had a "moderate pollution" record 3 years ago which has been cleaned, and the maximum load current in the most recent year was 500 amperes (A). Environmental sensor data shows that the average relative humidity in the most recent month was 85% RH, and the monthly equivalent salt deposit density (ESDD) was 0.15 mg / cm². This data is integrated into a feature vector as the insulator's operational context characteristics.

[0115] The insulator operating context features and recognition results are input together into a dynamic threshold evaluation model built on an ensemble learning framework.

[0116] Among them, the ensemble learning framework is a machine learning paradigm that completes learning tasks by building and combining multiple weak learners (such as decision trees), and usually achieves better generalization performance than a single learner; the dynamic threshold evaluation model specifically refers to a machine learning model trained based on the ensemble learning framework that can take contextual features and recognition results as input and output a floating discrimination threshold.

[0117] Specifically, the system can concatenate the insulator's operational context features (such as numerical features like years of operation, ambient humidity, and number of historical defects) with key information from the identification results (such as the identified anomaly category and its confidence level) to form a combined feature vector. This combined feature vector is then input into a pre-trained dynamic threshold evaluation model. This model can be a regression model built using a Gradient Boosting Decision Tree (GBDT) algorithm (such as XGBoost or LightGBM). For example, the system encodes and concatenates the context features [years of operation = 8, average monthly humidity = 85%, historical defects = 1, salt density = 0.15] with the identification results [anomaly category = "dirty", confidence level = 0.88]. Assuming the "dirty" category is encoded as 2, the combined feature vector might be: The vector is then fed into a trained XGBoost model.

[0118] The forward inference is performed by a dynamic threshold evaluation model, which outputs a dynamic discrimination threshold that is adapted to the current operating conditions and anomaly types.

[0119] Forward inference refers to the process of using input data to calculate the output result through a pre-trained machine learning model network; dynamic discrimination threshold refers to the critical value calculated by the model in real time to determine whether the current anomaly is valid and its severity. This value changes dynamically with the input environment, device and recognition features.

[0120] Specifically, the system executes the forward inference process of the model. The dynamic threshold evaluation model calculates sequentially through multiple decision trees based on the input combination features, ultimately summing the outputs of each tree to obtain a continuous real value. This value is the dynamic discrimination threshold for this specific evaluation (specific insulator, specific anomaly category, specific confidence level under specific conditions). This threshold reflects the minimum confidence level required to determine if the anomaly is a "true positive" under the current comprehensive operating conditions. For example, for the combination features in the above example, the XGBoost model outputs a value of 0.82 after forward inference. This 0.82 is the dynamically calculated discrimination threshold for this insulator that has been operating for 8 years in a high-humidity, high-pollution environment, identified as having a "pollution" anomaly, and has a confidence level of 0.88. This means that under these operating conditions, the confidence level must be greater than 0.82 to be accepted.

[0121] The abnormal feature parameters in the identification results are compared and calculated with the dynamic discrimination threshold. Based on the predefined threshold interval mapping rules, the health status level of the insulator is determined.

[0122] Here, the abnormal feature parameter mainly refers to the confidence score of the abnormal category in the identification result; the threshold interval mapping rule is a set of predefined judgment logic that maps the difference or ratio between the confidence score and the dynamic threshold to discrete health status levels.

[0123] Specifically, the system first calculates the difference between the abnormal feature parameter (i.e., confidence level C) in the identification result and the dynamic discrimination threshold T. Then, the state level is determined according to a predefined threshold interval mapping rule. For example, the rule can be defined as: if If so, the level is "normal"; if If so, the level is "slight abnormality"; if If so, the level is "moderately abnormal"; if If so, the level is "severely abnormal". and This is a fixed level span parameter. For example, if the confidence level C for "dirty" in the identification result is 0.88 and the dynamic discrimination threshold T is 0.82, then... Assuming the preset rules are as follows, .because ,satisfy Based on the given conditions, the system determines that the insulator's health status level in response to the "pollution" abnormality is "moderately abnormal".

[0124] Based on the determined health status level, the anomaly category label in the identification results, and the spatial coordinates of the anomaly location, the data is serialized and encapsulated into a structured comprehensive insulator status report containing anomaly descriptors, risk level codes, and maintenance recommendation codes.

[0125] Among them, serialization encapsulation refers to the process of combining various types of information into a complete data structure or document according to a fixed data format and order; the exception descriptor is a textual description of the identified exception; the risk level code is a standardized code representing the health status level (such as using numbers 1 to 4 to represent normal, minor, moderate, and severe, respectively); and the maintenance suggestion code is a standardized instruction code generated based on the health status level and exception category, which prompts subsequent operation and maintenance actions.

[0126] Specifically, the system can create a standardized report template (e.g., using JSON or XML format). It converts the determined health status level into the corresponding risk level code, uses the anomaly category label as the anomaly descriptor, and fills in the spatial coordinates (longitude, latitude, and altitude) of the anomaly location. Simultaneously, based on the "level-category" combination, the system queries a predefined maintenance strategy library to obtain the corresponding maintenance suggestion code (e.g., "A1" indicates "planned maintenance within 3 months," and "B2" indicates "emergency handling within 1 week"). Finally, all fields are filled in according to a predetermined order to generate the final structured insulator comprehensive status report. For example, the system generates a report where the anomaly descriptor is "pollution," the risk level code is "3" (representing moderate anomaly), and the maintenance suggestion code is "A1." The report also includes the spatial coordinates of the pollution point (e.g., E118.123456°, N32.654321°, H85.5m). This information is encapsulated in a JSON object: {"asset_id":"Tower-25-PhaseC","anomaly":"dirty","risk_level":3,"maintenance_code":"A1","location":{"lon":118.123456,"lat":32.654321,"alt":85.5},"confidence":0.88}. This JSON report is the Structured Insulator Comprehensive Status Report.

[0127] Therefore, according to the above implementation method, the system can go beyond simple judgment based on fixed thresholds. By introducing contextual features that reflect individual differences in equipment and the operating environment, it dynamically and individually determines the discrimination criteria using a machine learning model. This makes the state classification results more closely reflect the actual health status of the insulators, avoiding misjudgments that may result from a "one-size-fits-all" assessment (such as being overly harsh or lenient on old equipment or equipment in harsh environments). The resulting structured report provides maintenance personnel with direct, clear, and actionable decision support, enhancing the utilization value of inspection data and the level of intelligence in maintenance work.

[0128] In some embodiments, the steps of performing multi-dimensional data synchronous acquisition and spatiotemporal alignment are performed by the airborne sensing subsystem carried on the UAV platform; The steps of signal preprocessing and feature encoding of electric field measurement data at the airborne edge computing node, as well as the step of performing feature-level fusion at the airborne edge computing node, are executed by the edge computing subsystem mounted on the same platform.

[0129] Among them, the airborne sensing subsystem refers to the set of hardware integrated and installed on the UAV body, which is responsible for the perception of information in the physical world, including multi-physics field sensing arrays, high-definition visual sensors and their synchronous triggering and control circuits; the edge computing subsystem refers to the embedded computer module with certain computing capabilities mounted on the same platform, which is responsible for performing real-time or near-real-time data processing tasks near the data generation source.

[0130] Specifically, under the coordination of the UAV flight control system, the airborne sensing subsystem synchronously acquires and labels electric field, infrared, temperature and humidity, and image data. The acquired raw data is transmitted to the edge computing subsystem in real time via an internal bus. Upon receiving the data, the edge computing subsystem immediately invokes its internally deployed signal processing and neural network models to sequentially complete computationally intensive tasks such as electric field data preprocessing, feature encoding, and cross-modal feature fusion. For example, a hexacopter UAV carries an airborne sensing subsystem including a fiber optic electric field sensor, an uncooled infrared thermal imager, a temperature and humidity composite sensor, and a 4K industrial camera. It also carries an edge computing subsystem, the core of which is an NVIDIA Jetson AGX Orin embedded AI module. During inspection, the raw data acquired by the sensing subsystem is transmitted to the Jetson module via the PCIe (Peripheral Component Interconnect Express) bus, where the module completes all onboard computing tasks.

[0131] The step of transmitting the multi-dimensional fused feature representation to the ground computing node is executed via a high-speed air-space-ground wireless data link.

[0132] Among them, the high-speed air-space-ground wireless data link refers to a wireless network system used to establish a stable, high-bandwidth, low-latency communication connection between unmanned aerial vehicles (UAVs), ground stations (ground), and possible satellite relays (space).

[0133] Specifically, the system uses a high-speed air-to-ground wireless data link to transmit a significantly simplified multi-dimensional fused feature representation (relative to the original video stream) generated by the edge computing subsystem from a flying drone to a fixed ground receiving station in real time. This data link needs to overcome challenges such as drone movement and terrain obstruction to ensure the reliability and real-time performance of data transmission. For example, the system employs a high-speed air-to-ground wireless data link based on a 5G (fifth-generation mobile communication technology) private network or enhanced Wi-Fi 6 (sixth-generation wireless local area network technology). The drone establishes a connection with a ground base station deployed near the inspection area via an onboard 5G CPE (Customer Premises Equipment) or a high-performance Wi-Fi 6 network card, continuously providing uplink bandwidth of over 100 Mbps (megabits per second) to ensure that the fused feature representation, ranging from tens to hundreds of KB (kilobytes), is transmitted back to the ground within seconds.

[0134] The inference steps, which utilize lightweight target detection models deployed on ground computing nodes, are carried out and executed by a cluster of ground-based computing servers.

[0135] Among them, a computing server cluster refers to a hardware collection consisting of multiple high-performance servers interconnected through a network, used to provide centralized large-scale computing resources, which can be deployed inside a ground control station or in a cloud data center.

[0136] Specifically, the ground receiving station forwards the received multi-dimensional fused feature representation to the computing server cluster. A pre-trained lightweight object detection model is deployed in the cluster. The cluster scheduling system allocates inference tasks to server nodes equipped with GPUs (Graphics Processing Units), leveraging their powerful parallel computing capabilities to quickly complete deep analysis of the fused features and output recognition and localization results. For example, the computing server cluster consists of two servers equipped with NVIDIA RTX 4090 GPUs. The cluster deploys a lightweight YOLOv8 model optimized based on TensorRT. When a fused feature representation is received, the scheduler assigns it to one of the servers, which can complete a model inference within 50ms and output the anomaly recognition result for the insulator.

[0137] The steps of performing status classification based on the identification results and generating a comprehensive status report for structured insulators are executed by the operation and maintenance decision-making terminal that communicates with the computing server cluster.

[0138] Among them, the operation and maintenance decision terminal refers to a computing device with a graphical human-computer interaction interface for operation and maintenance personnel to receive and display processing results and support the generation of operation and maintenance instructions. It is usually a workstation computer, laptop computer or mobile terminal.

[0139] Specifically, after completing target detection and inference, the computing server cluster sends the identification results to a designated operation and maintenance decision-making terminal. The evaluation software running on this terminal invokes a dynamic threshold evaluation model, combines it with insulator operating context features queried from the database, completes state classification, and automatically populates all information, including the classification results, into a standard report template to generate the final evaluation report, which is then presented to the operation and maintenance personnel. For example, the operation and maintenance decision-making terminal is a workstation computer located in the inspection and control center. It obtains the information of "Insulator A, coordinates..." from the computing server cluster via the local area network. The system identified the insulator as 'damaged' with a confidence level of 0.95. The terminal software then retrieved the insulator's historical data from the database, calculated the dynamic threshold using the locally deployed XGBoost evaluation model, determined the status to be "severely abnormal," and automatically generated a PDF (Portable Document Format) report containing detailed location, images, severity level, and maintenance recommendations ("emergency replacement within 24 hours") for the maintenance team leader to review and issue a work order.

[0140] Among them, the airborne sensing subsystem and the edge computing subsystem are interconnected through the airborne high-speed data bus; the edge computing subsystem and the computing server cluster establish a communication session through a high-speed air-space-ground wireless data link; and the computing server cluster and the operation and maintenance decision terminal exchange data and transmit commands through an internal local area network or a dedicated network.

[0141] Among them, the airborne high-speed data bus refers to the physical communication link inside the UAV used to connect various sensors, computing units and flight control computer, requiring high bandwidth and low latency; the internal local area network or leased line network refers to the private wired network built in the ground control center or within the enterprise, used to connect servers, storage devices and terminals, to ensure the security and stability of data transmission.

[0142] Specifically, within the UAV platform, the sensors of the onboard sensing subsystem stream the collected data to the edge computing subsystem via an onboard high-speed data bus (such as Ethernet or Camera Link). Between the UAV and the ground, a point-to-point or network-connected communication session is established via a high-speed air-to-ground wireless data link to transmit control commands and computation results. On the ground side, the computing server cluster, database server, and operation and maintenance decision-making terminal are all connected to the same internal local area network, exchanging data via TCP / IP (Transmission Control Protocol / Internet Protocol). For example, the UAV uses Gigabit Ethernet as its onboard high-speed data bus to connect all sensors and the Jetson computing module. A dedicated 5G network serves as the high-speed air-to-ground wireless data link between the UAV and the ground. Inside the ground control center, all servers and terminals are interconnected via 10-gigabit fiber optic switches, forming a high-speed and reliable internal local area network.

[0143] Therefore, based on the above implementation method, the system can construct a well-structured and highly efficient four-level processing architecture: "sensor-edge-cloud-terminal". This architecture rationally distributes the computing load across different nodes: the airborne sensing subsystem is responsible for accurate perception, the edge computing subsystem is responsible for real-time preprocessing and feature fusion to reduce transmission pressure, the ground computing server cluster is responsible for complex model inference, and the operation and maintenance decision terminal is responsible for final decision-making and interaction. Each node is connected via optimized dedicated links, achieving fully automated, highly reliable, and low-latency operation from data acquisition to status report generation, meeting the stringent requirements of real-time performance, accuracy, and reliability for intelligent UAV inspection of power transmission lines.

[0144] Figure 6 This is a structural block diagram of a data processing system for unmanned aerial vehicle (UAV) inspection of insulators according to an embodiment of the present invention.

[0145] like Figure 6 As shown, the data processing system for the UAV insulator inspection includes: The data stream acquisition module 210 is used to perform multi-dimensional data synchronous acquisition and spatiotemporal alignment to acquire the multi-physics field sensing data stream and multi-angle image data stream of the insulator.

[0146] The feature vector output module 220 is used to perform signal preprocessing and feature encoding on electric field measurement data in multiphysics sensing data stream at an airborne edge computing node, and output electric field distribution feature vector.

[0147] The feature representation generation module 230 is used to input the electric field distribution feature vector, non-electric field modal data in the multi-physics sensing data stream and multi-angle image data stream into the cross-modal feature fusion network at the airborne edge computing node, and generate a unified multi-dimensional fused feature representation through feature-level fusion based on attention weights.

[0148] The spatial coordinate output module 240 is used to transmit the multi-dimensional fused feature representation to the ground computing node, and use the lightweight target detection model deployed on the ground computing node to infer the multi-dimensional fused feature representation, and output the identification result of the anomaly category on the insulator surface and the spatial coordinates of the anomaly location.

[0149] The status report generation module 250 is used to generate a comprehensive status report for structured insulators by performing status grading through a dynamic threshold evaluation model based on the identification results and the spatial coordinates of the abnormal location, combined with the insulator operating context features.

[0150] The specific functions and examples of each module and submodule of the device in this embodiment of the invention can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0151] According to embodiments of the present invention, the above-described method of the present invention can be applied to an electronic device and a readable storage medium.

[0152] Figure 7 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0153] like Figure 7 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0154] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0155] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as a data processing method for drone inspection of insulators. For example, in some embodiments, a data processing method for drone inspection of insulators can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the data processing method for drone inspection of insulators described above can be performed. Alternatively, in other embodiments, the computing unit 601 may be configured by any other suitable means (e.g., by means of firmware) to perform a data processing method for inspecting insulators by a drone.

[0156] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0157] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0158] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0159] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0160] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0161] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0162] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0163] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data processing method for unmanned aerial vehicle (UAV) inspection of insulators, characterized in that, include: Perform multi-dimensional data synchronous acquisition and spatiotemporal alignment to obtain multi-physics field sensing data streams and multi-angle image data streams of insulators; At the airborne edge computing node, the electric field measurement data in the multiphysics sensing data stream is preprocessed and feature encoded to output an electric field distribution feature vector. At the airborne edge computing node, the electric field distribution feature vector, the non-electric field modal data in the multi-physics sensing data stream, and the multi-angle image data stream are input into the cross-modal feature fusion network. Through feature-level fusion based on attention weights, a unified multi-dimensional fusion feature representation is generated. The multi-dimensional fusion feature representation is transmitted to the ground computing node, and the lightweight target detection model deployed on the ground computing node is used to infer the multi-dimensional fusion feature representation, and output the identification result of the anomaly category on the insulator surface and the spatial coordinates of the anomaly location. Based on the identification results and the spatial coordinates of the abnormal location, combined with the insulator operating context features, a structured insulator comprehensive status report is generated by evaluating the state classification of the dynamic threshold assessment model.

2. The method according to claim 1, characterized in that, The process of performing multi-dimensional data synchronous acquisition and spatiotemporal alignment to obtain multi-physics field sensing data streams and multi-angle image data streams of insulators includes: By deploying a multi-physics field sensing array on a drone platform, the raw electric field signal, infrared thermal radiation signal, and microenvironment temperature and humidity signal of the insulator are collected concurrently and aggregated to generate raw multimodal sensing data. Using a high-definition vision sensor deployed on the same platform, and based on preset track points and observation attitude, a multi-view image frame sequence of the insulator is acquired. For each data unit in the original multimodal sensing data and the multi-view image frame sequence, inject a unified time stamp and a coordinate label based on spatial positioning. Using the injected timestamps and coordinate labels as indexes, the original multimodal sensing data and the multi-view image frame sequence are spatiotemporally registered and aligned, and spatiotemporally synchronized multiphysics sensing data stream and multi-angle image data stream are output.

3. The method according to claim 1, characterized in that, The step of performing signal preprocessing and feature encoding on the electric field measurement data in the multiphysics sensing data stream at the airborne edge computing node, and outputting an electric field distribution feature vector, includes: Adaptive filtering is performed on the input electric field measurement data stream to suppress environmental noise and random interference, and the noise-reduced electric field time-domain signal is output. A multi-scale time-frequency transform is performed on the denoised electric field time-domain signal to extract the multi-scale frequency domain components. Local feature response enhancement is performed on the multi-scale frequency domain components to strengthen the feature response of the distortion region in each signal component; Feature embedding and dimensionality reduction are performed on the enhanced feature response to encode and generate a low-dimensional feature vector. This vector integrates the characteristics of electric field distortion, field strength gradient and phase shift, and serves as the electric field distribution feature vector.

4. The method according to claim 1, characterized in that, At the airborne edge computing node, the electric field distribution feature vector, the non-electric field modal data from the multi-physics sensing data stream, and the multi-angle image data stream are input into a cross-modal feature fusion network. Through feature-level fusion based on attention weights, a unified multi-dimensional fused feature representation is generated, including: The electric field distribution feature vector, the non-electric field modal data, and the multi-angle image data stream are respectively input into an independent modal feature encoder to extract electric field coding features, non-electric field coding features, and visual coding features; Based on the signal quality and data integrity corresponding to the electric field coding feature, the non-electric field coding feature and the visual coding feature, the reliability score of each modal feature is calculated, and the modal attention weight is dynamically allocated according to the reliability score. Using the pre-assigned modal attention weights, element-wise weighted summation is performed on the electric field encoded features, the non-electric field encoded features, and the visual encoded features to obtain preliminary fusion features; The preliminary fused features are concatenated with the original encoded features of each modality using tensors, and then transformed and compressed through a fully connected layer to output the unified multi-dimensional fused feature representation.

5. The method according to claim 1, characterized in that, The process of transmitting the multi-dimensional fused feature representation to a ground computing node, using a lightweight target detection model deployed on the ground computing node to infer the multi-dimensional fused feature representation, and outputting the identification result of the insulator surface anomaly category and the spatial coordinates of the anomaly location includes: The multi-dimensional fused feature representation is input into the backbone network of the lightweight target detection model, and forward propagation and feature mapping are performed through the depthwise separable convolutional layers and inverse residual structures in the backbone network. The feature map output by the backbone network is input into the coordinate attention module of the lightweight object detection model to recalibrate the spatial dimension weights of the feature map. The weighted and recalibrated feature map is input into the detection head of the lightweight target detection model; The adaptive anchor box generation layer and the classification regression layer of the detection head are used for parallel computation to output a prediction result containing anomaly category label, category confidence and target bounding box coordinates, where the target bounding box coordinates are the spatial coordinates of the anomaly location.

6. The method according to claim 1, characterized in that, Based on the identification results and the spatial coordinates of the abnormal locations, combined with the insulator's operational context features, a structured insulator comprehensive status report is generated by evaluating the state classification through a dynamic threshold assessment model, including: Load the insulator's full lifecycle operation data and environmental sensor data as the insulator's operating context features; The insulator operating context features and the identification results are input together into the dynamic threshold evaluation model built based on the ensemble learning framework; The dynamic threshold evaluation model performs forward inference and outputs a dynamic discrimination threshold that is adapted to the current working condition and the type of abnormality. The abnormal feature parameters in the identification results are compared and calculated with the dynamic discrimination threshold. Based on the predefined threshold interval mapping rules, the health status level of the insulator is determined. Based on the determined health status level, the anomaly category label in the identification result, and the spatial coordinates of the anomaly location, the data is serialized and encapsulated into a structured insulator comprehensive status report containing an anomaly descriptor, risk level code, and maintenance recommendation code.

7. The method according to claim 1, characterized in that, The step of performing multi-dimensional data synchronous acquisition and spatiotemporal alignment is executed by the airborne sensing subsystem carried on the UAV platform; The steps of performing signal preprocessing and feature encoding on the electric field measurement data at the airborne edge computing node, and the steps of performing feature-level fusion at the airborne edge computing node, are executed by the edge computing subsystem mounted on the same platform. The step of transmitting the multi-dimensional fused feature representation to the ground computing node is performed via a high-speed air-space-ground wireless data link. The step of inference using a lightweight target detection model deployed on the ground computing nodes is carried out and executed by a cluster of computing servers located on the ground. The step of performing state classification based on the identification results and generating the structured insulator comprehensive status report is executed by the operation and maintenance decision terminal that communicates with the computing server cluster; The airborne sensing subsystem and the edge computing subsystem are interconnected via an airborne high-speed data bus; the edge computing subsystem and the computing server cluster establish a communication session via the high-speed air-ground wireless data link; and the computing server cluster and the operation and maintenance decision terminal exchange data and transmit instructions via an internal local area network or a dedicated network.

8. A data processing system for unmanned aerial vehicle (UAV) inspection of insulators, characterized in that, include: The data stream acquisition module is used to perform multi-dimensional data synchronous acquisition and spatiotemporal alignment, and to acquire the multi-physics field sensing data stream and multi-angle image data stream of the insulator; The feature vector output module is used to perform signal preprocessing and feature encoding on the electric field measurement data in the multiphysics sensing data stream at the airborne edge computing node, and output the electric field distribution feature vector. The feature representation generation module is used to input the electric field distribution feature vector, the non-electric field modal data in the multi-physics sensing data stream and the multi-angle image data stream into the cross-modal feature fusion network at the airborne edge computing node, and generate a unified multi-dimensional fusion feature representation through feature-level fusion based on attention weights. The spatial coordinate output module is used to transmit the multi-dimensional fused feature representation to the ground computing node, and use the lightweight target detection model deployed on the ground computing node to infer the multi-dimensional fused feature representation, and output the identification result of the anomaly category on the insulator surface and the spatial coordinates of the anomaly location. The status report generation module is used to generate a comprehensive status report for the structured insulator based on the identification results and the spatial coordinates of the abnormal location, combined with the insulator operating context features, and to perform status grading through a dynamic threshold evaluation model.

9. An electronic device, characterized in that, include: At least one processor; and a memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, in, Computer instructions are used to cause a computer to perform the method according to any one of claims 1-7.

Images