A method and system for real-time inspection of power safety of unmanned aerial vehicles
Patent Information
- Application Number
- CN202611032179.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-13
- Publication Date
- 2026-10-16
AI Technical Summary
这种模式存在明显的滞后性，无法在巡检过程中及时发现和处理线路隐患
[0034] (1) The UAV power safety real-time inspection method and system of the present invention can detect diverse targets and simultaneously and accurately identify defects such as insulator damage, metal component corrosion/missing/deformation, as well as foreign objects such as bird nests, kites, and plastic bags, achieving full-scene coverage inspection of transmission lines. It has high detection accuracy. Through the collaborative optimization of multiple modules such as IDC, SEAM, and EMA, it effectively solves the detection problems of small targets, obscured targets, and complex backgrounds, and improves the precision and recall rates.
Smart Images
Figure CN122929430A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of unmanned aerial vehicle (UAV) inspection technology, and in particular to a method and system for real-time inspection of power safety using UAVs. Background Technology
[0002] Power lines are a crucial component of modern power systems and future smart grids, and their stable and safe operation is essential for the normal functioning of the entire power system. Damage to components such as conductors, insulators, and fittings, as well as damage caused by natural disasters, weather conditions, and external forces, can all lead to unplanned outages of overhead lines, causing serious disruptions to power supply. As the scale and complexity of power grids continue to expand, the inspection tasks of transmission lines are becoming increasingly demanding, requiring higher efficiency, accuracy, and real-time performance in inspections.
[0003] Traditional power transmission line inspections primarily rely on manual methods. Inspectors must carry testing equipment and patrol along the lines on foot or by vehicle, observing the condition of the lines and components with the naked eye or binoculars. This method suffers from low efficiency, incomplete coverage, and high safety risks in complex terrain environments such as mountains, dense forests, rivers, and swamps. Furthermore, it is greatly affected by weather conditions, making it difficult to achieve high-frequency, all-weather inspections. At the same time, manual inspections depend on the experience and judgment of the inspectors, are highly subjective, and are prone to missed or false detections, especially in their limited ability to identify early, minor defects.
[0004] With the rapid development of drone technology, drone inspection has gradually become an important auxiliary means for power transmission line inspection. However, existing drone inspections are mostly reactive, involving drones carrying cameras to capture images of the lines, exporting the image data after the drone returns, and then manually screening and analyzing the results. This model has significant time lags, failing to promptly identify and address potential line hazards during the inspection process. Furthermore, as inspection tasks increase, the amount of image data collected grows exponentially, making manual screening in complex environments ineffective and inefficient, failing to meet the requirements of smart grids for real-time monitoring and rapid response.
[0005] Regarding detection algorithms, existing UAV inspection systems suffer from the following shortcomings: First, insufficient detection accuracy, with poor ability to identify small targets (such as minor defects and early damage) and targets in complex backgrounds (such as occluded targets and targets similar to the background), making it difficult to meet the requirements of refined management of power transmission lines; Second, high model complexity, with a large number of parameters and computational load, making it difficult to achieve real-time inference operation under the limited computing resources and power consumption of UAVs, usually requiring images to be transmitted back to ground workstations for processing, thus failing to achieve real-time airborne detection; Third, insufficient robustness of the algorithms to environmental and lighting changes, with significant performance degradation in complex environments such as backlight, fog, and shadows.
[0006] Furthermore, existing UAV inspection systems also have limitations in terms of hardware platforms. The embedded platforms on ordinary UAVs have limited computing power, making it difficult to support the real-time operation of complex deep learning models; while high-performance computing platforms often have high power consumption and large size, affecting the UAV's endurance and flight performance. How to achieve lightweight models and efficient hardware deployment while ensuring detection accuracy has become a key issue restricting the development of UAV power line inspection technology. Summary of the Invention
[0007] This invention provides a method and system for real-time inspection of power safety using unmanned aerial vehicles (UAVs), which can improve detection accuracy, achieve lightweight deployment, and meet the needs of real-time inspection.
[0008] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:
[0009] A method for real-time inspection of power safety using unmanned aerial vehicles (UAVs) includes the following steps:
[0010] Step S1: Construct a lightweight, high-precision power line inspection and detection model. The detection model includes a backbone feature extraction network, a feature enhancement module, a feature fusion module, and a detection head.
[0011] Step S2: Acquire real-time images of the power transmission line to be operated using a drone, extract key frames from the real-time images, and input the extracted key frames into the detection model.
[0012] Step S3: The backbone feature extraction network extracts features from the keyframes to obtain multi-scale initial feature information;
[0013] Step S4: The feature enhancement module performs multi-dimensional enhancement processing on the initial features;
[0014] Step S5: The feature fusion module performs cross-scale interactive fusion of the enhanced multi-scale features;
[0015] Step S6: The detection head performs target classification and localization on the fused features and outputs the detection results of foreign objects or component defects in the transmission line.
[0016] Step S7: The detection results, abnormal images, and corresponding geographical locations and inspection path information are transmitted back to the cloud storage server in real time, and the inspection data is displayed through the client.
[0017] A further improvement to the above technical solution is as follows:
[0018] Preferably, the backbone feature extraction network is based on YOLOv11 architecture and includes a C3K2 module, a Bottleneck Block module, and a C2PSA module; the feature enhancement module includes an IDC module, a SEAM module, and an EMA module; the feature fusion module includes an AIFI module; and the detection head is an anchorless design and includes a classifier and a regressor.
[0019] Preferably, the IDC module includes a first branch, a second branch, a third branch, and a fourth branch configured in parallel. The first branch is a 3×3 convolution, the second branch is a 1×11 convolution, the third branch is an 11×1 convolution, and the fourth branch is an identity mapping. The outputs of the first branch, the second branch, the third branch, and the fourth branch are concatenated along the channel dimension to serve as the output of the IDC module.
[0020] Preferably, when the input feature map is segmented along the channel dimension, the number of channels in the first branch, the second branch, and the third branch is: ,in The total number of channels in the input feature map. As the branch ratio, the number of channels in the fourth branch is .
[0021] Preferably, the SEAM module includes a multi-view feature self-integration unit and a CSMM module; the multi-view feature self-integration unit compares the consistency of the prediction results of the original image and the transformed image through a Siamese network architecture; the CSMM module includes a patch embedding layer, a depthwise separable convolutional layer, a point convolutional layer and an activation layer, and the CSMM module adopts depthwise separable convolution.
[0022] Preferably, in the SEAM module, the input feature map is fed in parallel into multiple CSMM modules with different patch sizes for processing, and the multi-scale feature maps output by each branch are fused by adding them element by element.
[0023] Preferably, the EMA module includes: a channel splitting unit, a parallel branching unit, and a cross-dimensional interactive aggregation branch; the channel splitting unit is used to split the input feature map into G sub-feature groups along the channel dimension, and the number of channels in each sub-feature group is... The parallel branch unit includes two 1×1 convolutional branches and one 3×3 depthwise convolutional branch; the cross-dimensional interactive aggregation branch is used to aggregate the output of the parallel branch unit, generate a spatial attention map, and perform weighted aggregation on the output of the sub-feature groups.
[0024] Preferably, the AIFI module is used to perform internal scale feature interaction on the high-level feature layer, which is an S5 level feature layer.
[0025] The present invention also provides a real-time power safety inspection system for unmanned aerial vehicles (UAVs), comprising:
[0026] The model building module is used to build a lightweight and high-precision power line inspection and detection model. The detection model includes a backbone feature extraction network, a feature enhancement module, a feature fusion module, and a detection head.
[0027] The image acquisition module is used to acquire images of the power transmission line to be operated in real time using a drone equipped with a visible light camera and a Yulong810A chip, perform key frame extraction processing on the real-time images, and input the extracted key frames into the detection model.
[0028] The feature extraction module is used to extract features from the keyframes through the backbone feature extraction network to obtain multi-scale initial feature information;
[0029] The feature enhancement module is used to perform multi-dimensional enhancement processing on the initial features;
[0030] The feature fusion module is used to perform cross-scale interactive fusion of enhanced multi-scale features;
[0031] The detection module is used to classify and locate targets based on the fused features through the detection head, and output the detection results of foreign objects or component defects in the transmission line.
[0032] The communication and display module is used to transmit the detection results, abnormal images, and corresponding geographical location information and inspection path information back to the cloud storage server in real time via the TCP protocol, and display them through the client.
[0033] The real-time power safety inspection method and system for unmanned aerial vehicles (UAVs) provided by this invention have the following advantages compared with existing technologies:
[0034] (1) The UAV power safety real-time inspection method and system of the present invention can detect diverse targets and simultaneously and accurately identify defects such as insulator damage, metal component corrosion / missing / deformation, as well as foreign objects such as bird nests, kites, and plastic bags, achieving full-scene coverage inspection of transmission lines. It has high detection accuracy. Through the collaborative optimization of multiple modules such as IDC, SEAM, and EMA, it effectively solves the detection problems of small targets, obscured targets, and complex backgrounds, and improves the precision and recall rates.
[0035] (2) The UAV power safety real-time inspection method and system of the present invention has a significant effect on model lightweighting. The AIFI module only performs self-attention interaction on the high-level feature layer to avoid redundant calculation of low-level features. The IDC module adopts grouped multi-branch small kernel decomposition to reduce the quadratic complexity of large kernel convolution to linear complexity. Combined with the hardware acceleration capability of Yulong810A chip, the computational complexity and number of parameters are greatly reduced, which meets the real-time inspection requirements of UAV under limited computing resources.
[0036] (3) The real-time inspection method and system for power safety of UAVs of the present invention are highly practical. The detection device is based on a visible light camera and a Yulong810A chip, which is easy to install on UAVs of different specifications (fixed-wing aircraft, small and medium-sized UAVs), adaptable to complex inspection environments, and the chip can work in extreme temperatures of -55℃ to 125℃, adapting to various climatic conditions. Attached Figure Description
[0037] Figure 1 This is the overall architecture diagram of the inspection system of this invention;
[0038] Figure 2 This is a flowchart of the YOLOv11 basic algorithm of the detection model of this invention;
[0039] Figure 3 This is a schematic diagram of the working principle of the IDC module of this invention;
[0040] Figure 4 This is a schematic diagram of the working principle of the AIFI module of this invention;
[0041] Figure 5 This is a schematic diagram of the working principle of the EMA module of this invention. Detailed Implementation
[0042] The following provides a detailed description of specific embodiments of the present invention. It should be understood that the specific embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the scope of the invention.
[0043] This invention provides a method and system for real-time power safety inspection using unmanned aerial vehicles (UAVs). The overall technical solution includes eight core steps: model construction, image acquisition and keyframe extraction, multi-scale feature extraction, multi-dimensional feature enhancement, cross-scale feature fusion, target detection output, data feedback and display, and model optimization and deployment. The method builds a lightweight detection model based on the YOLOv11 architecture. The IDC module decomposes a large-kernel deep convolutional layer into four parallel branches to expand the receptive field and reduce computational load. The SEAM module utilizes multi-view feature self-integration, and the CSMM module enhances the representation of occluded targets. The EMA module employs a cross-dimensional interactive aggregation mechanism to avoid channel information loss. The AIFI module performs self-attention interaction on high-level feature layers to improve fusion efficiency. The system uses the Yulong810A chip as the computing platform, supporting onboard real-time inference and cloud-based closed-loop data management. The following detailed description of each step is provided with reference to the accompanying drawings.
[0044] like Figure 1 As shown, the real-time power safety inspection method for drones of the present invention specifically includes the following steps:
[0045] Step S1: Construct a lightweight, high-precision power line inspection and detection model.
[0046] The model consists of a backbone feature extraction network, a feature enhancement module, a feature fusion module, and a detection head.
[0047] S1-1, Backbone Feature Extraction Network: Based on YOLOv11 (e.g., ... Figure 2 As shown, the C3K2 module is used as the core component, combined with the Bottleneck Block and C2PSA modules, to output multi-scale initial feature layers through multiple sets of convolutional layers and feature extraction units. In this embodiment, feature maps at three scales, P3, P4, and P5, are output, with sizes of 80×80 pixels, 40×40 pixels, and 20×20 pixels, and the number of channels are 256, 512, and 1024, respectively.
[0048] S1-2, Feature Enhancement Module: Composed of IDC module, SEAM module and EMA module.
[0049] S1-2-1, IDC module (Inception Depthwise Convolution) (e.g.) Figure 3 As shown): To overcome the inefficiency of large kernel depthwise convolutions (such as the 7×7 depthwise convolution in ConvNeXt) on actual hardware (high memory access cost, low throughput), this module decomposes a single large kernel depthwise convolution into four parallel branches, and groups the input feature maps along the channel dimension, which maintains a large effective receptive field and significantly improves computational efficiency.
[0050] Let the input feature map be ,in Batch size refers to the number of samples processed simultaneously at one time. The number of channels in the input feature map. The height is the feature map height. The width of the feature map. For the real number field. Define the number of branch channels. ,in The number of channels allocated to each processing branch. Branch scaling factor, default .
[0051] S1-2-1-1, first along the channel dimension... It is divided into four feature map groups:
[0052]
[0053] in, For the feature map set fed into the kernel-small convolution branch, and subscript This indicates that the branch processes height simultaneously. and width Two spatial dimensions; The feature map set fed into the horizontal band convolution branch, and subscript This indicates that the branch focuses on width. Direction (horizontal direction); To feed the feature map set into the vertical strip convolution branch, and subscript This indicates that the branch focuses on height. Direction (perpendicular direction); The feature map set fed into the identity mapping branch, and subscript This indicates an identity mapping, meaning that the data is passed directly without any transformation. For the real number field, For batch size, The number of channels in the input feature map. For feature map height, The width of the feature map. The number of channels allocated to each processing branch; This represents the number of channels in the fourth branch, where This represents the total number of channels occupied by the first three branches; Segmentation operation along the channel dimension.
[0054] S1-2-1-2, the four feature map groups are sent to different parallel branches for processing.
[0055] (1) Mini-kernel convolution branch:
[0056]
[0057] in, The output feature map of the small kernel convolution branch; For depthwise separable convolution, spatial convolution is first performed independently on each input channel, and then the channel information is mixed through point convolution. Indicates the kernel size. For small core size parameters, Indicates a small core; Indicates the number of input channels is The number of output channels is also The number of channels remains unchanged; This indicates that the input to the convolution operation is... .
[0058] (2) Horizontal band convolution branch:
[0059]
[0060] in, The output feature map of the horizontal strip convolution branch; For depthwise separable convolution, spatial convolution is first performed independently on each input channel, and then the channel information is mixed through point convolution. Indicates the number of input channels is The number of output channels is also ; This indicates the kernel size, with a height of 1 pixel and a width of [missing information]. Pixels It indicates a strip or band.
[0061] (3) Vertical band convolution branch:
[0062]
[0063] in, The output feature map of the vertical strip convolution branch; For depthwise separable convolution, spatial convolution is first performed independently on each input channel, and then the channel information is mixed through point convolution. Indicates the number of input channels is The number of output channels is also ; This represents the kernel size, with a height of [value missing]. Pixels, with a width of 1 pixel;
[0064] (4) Identity mapping branch:
[0065]
[0066] in, The output feature map of the identity mapping branch; The input feature map set is the identity mapping branch.
[0067] S1-2-1-3, concatenate the outputs of all branches along the channel dimension to obtain the enhanced feature map. :
[0068]
[0069] in, This is the final output feature map of the IDC module; This is a splicing operation along the channel dimension; The output feature map of the small kernel convolution branch. This is the output feature map of the horizontal band convolution branch. This is the output feature map of the vertical strip convolution branch. This is the output feature map of the identity mapping branch.
[0070] After concatenation, the feature map is output. The dimension is The number of channels was restored to .
[0071] S1-2-2, SEAM Module (Spatially Enhanced Attention Module): The SEAM module enhances robustness through multi-view feature fusion and consistency regularization. It contains the following two core substructures:
[0072] Multi-view feature self-integration unit: adopts a Siamese network architecture to compare the consistency of prediction results between the original image and the transformed image, and uses self-supervised signals to force the model to maintain the stability of target semantic understanding under viewpoint changes;
[0073] The CSMM (Channel-Space Mixed Module) consists of a Patch Embedding layer for segmenting and embedding the input feature map, a depthwise separable convolutional layer for independently extracting features in the spatial dimension, and a point convolutional and activation layer for channel relationship reconstruction. This module efficiently extracts multi-scale spatial and channel features while reducing the number of computational parameters. The self-integration of multi-view features compares the consistency of prediction results between the original and transformed images using a Siamese network architecture, and uses self-supervised signals to force the model to maintain stability in understanding the target semantics under changing viewpoints.
[0074] The specific fusion method is as follows: the input feature map is fed into multiple CSMM modules with different patch sizes in parallel for processing, and the multi-scale feature maps output by each branch are fused by adding them element by element to obtain feature representations that have both local details and global semantics.
[0075] The SEAM module further utilizes consistency regularization to suppress background interference, thereby improving the feature representation of occluded targets.
[0076] S1-2-3, EMA module (Efficient Multi-scale Attention): The EMA module is used to replace the ordinary attention mechanism in the C2PSA module to avoid information loss caused by channel dimension reduction.
[0077] The specific implementation steps are as follows:
[0078] S1-2-3-1, channel splitting.
[0079] The input feature map is split along the channel dimension. Each sub-feature group has a reduced number of channels. .in The number of groups is set to the default value in this embodiment. , The number of channels in the input feature map.
[0080] S1-2-3-2, Dimensional Reshaping.
[0081] The grouped channels are reshaped into batch dimensions. Let the original feature dimension be... After being grouped, it becomes By merging the group dimension and the batch dimension, we obtain This maps the original channel dimension to the batch dimension, avoiding information loss caused by channel dimensionality reduction.
[0082] S1-2-3-3, parallel branch subnetwork.
[0083] Construct a parallel structure containing two 1×1 convolutional branches and one 3×3 depthwise convolutional branch:
[0084] (1) 1×1 convolutional branch (height direction): Perform one-dimensional global average pooling along the height direction of the spatial dimension to encode inter-channel information. The feature shape after one-dimensional pooling is as follows: .
[0085] 1×1 convolutional branch (width direction): Performs one-dimensional global average pooling along the width direction of the spatial dimension, encoding inter-channel information. The feature shape after one-dimensional pooling is... .
[0086] Shared Convolution: The two 1×1 branches mentioned above share a single 1×1 convolution with no channel reduction, meaning that both the input and output channels of the convolution are equal. No dimensionality reduction is performed.
[0087] (2) 3×3 depthwise convolution branch: Use 3×3 depthwise convolution to capture multi-scale spatial feature representations, with both input and output channels having the same number of channels. .
[0088] S1-2-3-4, cross-dimensional interactive aggregation.
[0089] Aggregate the output features of the two 1×1 branches and the one 3×3 branch:
[0090] Perform two-dimensional global average pooling on the encoded output of the 1×1 branch to obtain a global spatial information descriptor with the shape of ;
[0091] Transform the output of the 3×3 branch into a dimension shape that matches the output of the 1×1 branch;
[0092] Two sets of spatial attention maps are generated through matrix dot product operations, with the following shape: ;
[0093] The generated spatial attention weights are used to weight and aggregate the output feature maps within each sub-feature group to obtain the final output, the shape of which is restored to [formula missing]. .
[0094] S1-3, Feature Fusion Module: The feature fusion module replaces the traditional SPPF module with the AIFI module.
[0095] like Figure 4As shown, the AIFI module (Attention-based Internal Feature Interaction) focuses on the internal scale feature interactions of the S5 high-level feature layer (20×20 pixels). S5 is the top-level feature map in the feature pyramid, possessing the smallest spatial size and the largest receptive field. The AIFI module captures the connections between semantic concepts through a self-attention mechanism, avoiding redundant computations caused by low-level feature interactions.
[0096] In this embodiment, the AIFI module employs a Transformer encoder structure, combining upsampling and downsampling operations from the FPN (Feature Pyramid Network) and PAN (Path Aggregation Network) modules to achieve deep fusion of multi-scale features. Upsampling transmits high-level semantic information to lower levels, allowing small-sized feature maps to be enlarged and fused with larger-sized feature maps. Downsampling transmits low-level positional information to higher levels, allowing large-sized feature maps to be reduced in size and fused with higher-level feature maps. The combination of these two methods forms a bidirectional feature pyramid.
[0097] S1-4, Detection head: adopts an anchorless design and consists of a classifier and a regressor.
[0098] The classifier outputs the probability of the target category, which includes categories such as insulator damage, hardware corrosion, bird nests, kites, and plastic film.
[0099] The regressor outputs the position coordinates of the target bounding box, including the x-coordinate of the center point, the y-coordinate of the center point, the width of the bounding box, and the height of the bounding box.
[0100] Step S2: Using a drone equipped with a high-resolution visible light camera (RGB camera) and a Yulong810A chip, real-time images of the power transmission line to be operated are acquired.
[0101] Set two adjacent frames of input and The image has a height and width of H×W, and the number of channels C=3 (stores RGB information). When the difference between two frames exceeds a preset threshold, the image is determined to have entered a new transmission line segment. The model has been updated to a keyframe input detection model to reduce redundant recognition and computational overhead. The difference calculation formula is as follows:
[0102]
[0103] in, This represents a measure of the difference between two images. This indicates the previous frame of the image, and this indicates the next frame of the image. Indicates the channel index. Represents the pixel row index, Indicates the pixel column index. Indicates the image height. Indicates the image width.
[0104] when hour( (Based on a preset threshold), if the image enters a new transmission line segment, then... The model has been updated to a keyframe input detection model to reduce redundant recognition and computational costs.
[0105] The type of drone can be selected according to the inspection environment. For example, fixed-wing drones are used in the desert areas of Northwest China, while small and medium-sized drones are used in the complex urban environment. The detection device is separated from the drone platform, which makes it highly adaptable.
[0106] The Yulong810A chip adopts a heterogeneous multi-core architecture, integrating a quad-core ARM Cortex-A55 CPU and a dedicated AI accelerator NPU, supporting hardware acceleration for deep learning algorithms, and can operate stably in an extreme temperature range of -55℃ to 125℃. The overall system power consumption is controlled below 15W, without affecting the drone's endurance.
[0107] Step S3: The backbone feature extraction network of the detection model extracts features from the keyframes.
[0108] The input image is first preprocessed by adaptive anchor box calculation, adaptive image scaling, and Mosaic data augmentation before entering the backbone network.
[0109] Four images were randomly scaled, cropped, and then stitched together into one. The image is calculated using the following formula:
[0110]
[0111] in, This represents the output image after Mosaic enhancement. This indicates an image scaling operation. This indicates a splicing operation. This represents four randomly selected original training images; Represents the random scaling factor for each of the four images, ranging from... .
[0112] Initial feature extraction is performed through multiple sets of convolutional layers, and then processed step by step using components such as the C3K2 module and the Bottleneck Block to output initial feature layers of different scales, providing a foundation for subsequent enhancement and fusion.
[0113] Step S4: The feature enhancement module enhances the initial features in multiple dimensions.
[0114] The IDC module extracts local and global features through a multi-branch parallel structure, improving its adaptability to small targets and complex backgrounds; the SEAM module enhances the feature representation of occluded targets and reduces the impact of background interference; the EMA module optimizes multi-scale feature extraction, enhancing feature discriminativeness, such as... Figure 5 As shown.
[0115] Enhanced process:
[0116]
[0117] in: This represents the initial feature map at the third scale (P3). This represents the initial feature map at scale 4 (P4). This represents the initial feature map at scale 5 (P5). express Feature map enhanced by IDC module express Feature maps enhanced by IDC+SEAM express The final feature map after IDC+SEAM+EMA enhancement express Feature maps enhanced by different modules express Feature maps enhanced by different modules.
[0118] Step S5: The feature fusion module performs cross-scale interactive fusion on the enhanced multi-scale features.
[0119] The AIFI module performs internal scale interaction on the S5 high-level feature layer to capture semantic relationships. Combined with the operations of the FPN and PAN modules, it performs upsampling and downsampling fusion of features at different scales to form rich, comprehensive, and fused features. Upsampling uses bilinear interpolation, and downsampling uses convolution with a stride of 2.
[0120] AIFI processing:
[0121]
[0122] FPN+PAN fusion:
[0123]
[0124] in, This represents the output of the AIFI module after processing the feature map of layer S5. This represents the feature map after top-down path (FPN) fusion. This represents the final feature map after bottom-up path (PAN) fusion.
[0125] Three sets of fusion features were finally obtained. .
[0126] In step S6, the detection head inputs the three sets of fused features, uses a classifier to determine the target category (such as insulator damage, bird nest, hardware corrosion, etc.), uses a regressor to determine the target location, and generates the final accurate detection output.
[0127] For each grid location on each feature map, the output is: class probability. and bounding box: ,in Represents the category probability vector. Indicates the total number of categories. Indicates the predicted bounding box parameters. This indicates the offset of the regression header output.
[0128] Convert the offset to actual coordinates:
[0129]
[0130] in, The coordinates of the top left corner of the grid. Indicates the actual center coordinates. Indicates the actual width and height; This represents the Sigmoid activation function, used to restrict the offset to the range (0,1); This indicates an exponential transformation of the width and height, ensuring the values are positive.
[0131] The final test results include: target category (insulator damage, bird nests, hardware corrosion, kites, plastic bags, etc.) and confidence score. ) and bounding box coordinates .
[0132] Step S7: The detection results (target category, confidence level), abnormal images, corresponding geographical location information and inspection path data are transmitted back to the cloud storage server in real time via TCP protocol.
[0133] The client is built on the VUE3 framework and includes both a web website and an app. It supports real-time viewing of basic information such as inspection anomaly photos, geographical location, and inspection route. After the inspection is completed, an inspection report is automatically generated, realizing data visualization and task management.
[0134] Step S8 involves optimizing and deploying the model structure. The reparameterization technique of the MobileOne network is used to merge the multi-branch structure into a single convolutional kernel, further reducing computational complexity. The model is exported to ONNX format using the torch.onnx.export function and deployed to the Yulong810A chip platform to achieve efficient real-time inference using the chip's AI acceleration capabilities.
[0135] The present invention provides a real-time power safety inspection system for unmanned aerial vehicles (UAVs), comprising:
[0136] Model building module: Used to build the above-mentioned lightweight and high-precision power line inspection and detection model.
[0137] Image acquisition module: Used to acquire real-time images of the power transmission line to be operated using a drone equipped with a visible light camera and Yulong810A chip, perform key frame extraction processing on the real-time images, and input the extracted key frames into the detection model.
[0138] Feature extraction module: Used to extract features from keyframes through the backbone feature extraction network to obtain multi-scale initial feature information.
[0139] Feature enhancement module: Used to perform multi-dimensional enhancement processing on the initial features (including IDC, SEAM, and EMA sub-modules).
[0140] Feature fusion module: used to perform cross-scale interactive fusion of enhanced multi-scale features (including AIFI, FPN, and PAN sub-modules).
[0141] Detection module: Used to classify and locate targets based on the fused features using a detection head, and output detection results.
[0142] Communication and Display Module: Used to transmit detection results, abnormal images, geographical locations, and inspection path information back to the cloud storage server in real time and display them through the client.
[0143] The above embodiments are merely preferred examples of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.
Claims
1. A method for real-time inspection of power safety using unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Step S1: Construct a lightweight, high-precision power line inspection and detection model. The detection model includes a backbone feature extraction network, a feature enhancement module, a feature fusion module, and a detection head. Step S2: Acquire real-time images of the power transmission line to be operated using a drone, extract key frames from the real-time images, and input the extracted key frames into the detection model. Step S3: The backbone feature extraction network extracts features from the keyframes to obtain multi-scale initial feature information; Step S4: The feature enhancement module performs multi-dimensional enhancement processing on the initial features; Step S5: The feature fusion module performs cross-scale interactive fusion of the enhanced multi-scale features; Step S6: The detection head performs target classification and localization on the fused features and outputs the detection results of foreign objects or component defects in the transmission line. Step S7: The detection results, abnormal images, and corresponding geographical locations and inspection path information are transmitted back to the cloud storage server in real time, and the inspection data is displayed through the client.
2. The method for real-time inspection of power safety by unmanned aerial vehicles according to claim 1, characterized in that... The backbone feature extraction network is based on YOLOv11 and includes a C3K2 module, a Bottleneck Block module, and a C2PSA module; the feature enhancement module includes an IDC module, a SEAM module, and an EMA module; the feature fusion module includes an AIFI module; and the detection head is an anchorless design and includes a classifier and a regressor.
3. The method for real-time inspection of power safety by unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The IDC module includes a first branch, a second branch, a third branch, and a fourth branch configured in parallel. The first branch is a 3×3 convolution, the second branch is a 1×11 convolution, the third branch is an 11×1 convolution, and the fourth branch is an identity mapping. The outputs of the first branch, the second branch, the third branch, and the fourth branch are concatenated along the channel dimension to serve as the output of the IDC module.
4. The method for real-time inspection of power safety by unmanned aerial vehicles according to claim 3, characterized in that, When the input feature map is segmented along the channel dimension, the number of channels in the first branch, the second branch, and the third branch is: ,in The total number of channels in the input feature map. As the branch ratio, the number of channels in the fourth branch is .
5. The method for real-time inspection of power safety by unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The SEAM module includes a multi-view feature self-integration unit and a CSMM module. The multi-view feature self-integration unit compares the consistency of the prediction results of the original image and the transformed image through a Siamese network architecture. The CSMM module includes a patch embedding layer, a depthwise separable convolutional layer, a point convolutional layer, and an activation layer, and the CSMM module adopts depthwise separable convolution.
6. The method for real-time inspection of power safety by unmanned aerial vehicles according to claim 5, characterized in that, In the SEAM module, the input feature maps are fed in parallel into multiple CSMM modules with different patch sizes for processing, and the multi-scale feature maps output by each branch are fused by adding them element by element.
7. The method for real-time inspection of power safety by unmanned aerial vehicles (UAVs) according to claim 6, characterized in that, The EMA module includes: a channel splitting unit, a parallel branching unit, and a cross-dimensional interactive aggregation branch; the channel splitting unit is used to split the input feature map into G sub-feature groups along the channel dimension, and the number of channels in each sub-feature group is G. The parallel branch unit includes two 1×1 convolutional branches and one 3×3 depthwise convolutional branch; the cross-dimensional interactive aggregation branch is used to aggregate the output of the parallel branch unit, generate a spatial attention map, and perform weighted aggregation on the output of the sub-feature group.
8. The method for real-time inspection of power safety by unmanned aerial vehicles according to claim 7, characterized in that, The AIFI module is used to perform internal scale feature interaction on the high-level feature layer, which is an S5 level feature layer.
9. A real-time power safety inspection system for unmanned aerial vehicles (UAVs), used to execute the real-time power safety inspection method for UAVs according to any one of claims 1 to 8, characterized in that, The inspection system includes: The model building module is used to build a lightweight and high-precision power line inspection and detection model. The detection model includes a backbone feature extraction network, a feature enhancement module, a feature fusion module, and a detection head. The image acquisition module is used to acquire images of the power transmission line to be operated in real time using a drone equipped with a visible light camera and a Yulong810A chip, perform key frame extraction processing on the real-time images, and input the extracted key frames into the detection model. The feature extraction module is used to extract features from the keyframes through the backbone feature extraction network to obtain multi-scale initial feature information; The feature enhancement module is used to perform multi-dimensional enhancement processing on the initial features; The feature fusion module is used to perform cross-scale interactive fusion of enhanced multi-scale features; The detection module is used to classify and locate targets based on the fused features through the detection head, and output the detection results of foreign objects or component defects in the transmission line. The communication and display module is used to transmit the detection results, abnormal images, and corresponding geographical location information and inspection path information back to the cloud storage server in real time via the TCP protocol, and display them through the client.