Unmanned aerial vehicle inspection method and device for tower components of power distribution network and unmanned aerial vehicle

Abstract

This invention provides a method, apparatus, and drone for unmanned aerial vehicle (UAV) inspection of power distribution network tower components, belonging to the field of power inspection. The method includes: acquiring a real-time image set and determining the identification information of the object to be inspected in the image set based on the image set; determining a first position of the UAV based on the position and size of the object in the visible light image and a standard image; acquiring first environmental data and a first multispectral image; the first multispectral image is taken by the UAV at the first position, and the first environmental data is data about the environment in which the UAV was located when taking the first multispectral image; correcting the spectral reflectance of the object in the first multispectral image based on the first environmental data and the environmental data when the standard multispectral image was taken, obtaining a first spectral reflectance curve of the object; and inspecting the object based on the first spectral reflectance curve and the standard spectral reflectance curve. This invention can improve the accuracy of UAV inspection.

Description

Technical Field

[0001] This invention relates to the field of power grid inspection technology, and in particular to a method, apparatus and drone for unmanned aerial vehicle (UAV) inspection of pole components in power distribution networks. Background Technology

[0002] With the development of technology, drones are being used more and more widely in the power industry, especially in the field of distribution network inspection. Drones can complete a large number of pole and tower inspections in a short time, improve work safety, and greatly reduce the time and manpower costs required for inspections.

[0003] Drones, equipped with gimbal cameras, can accurately identify power grid towers and their various components. This includes identifying critical parts such as the tower body, conductors, insulators, and grounding wires to ensure they are in normal operating condition. They can also detect anomalies in the towers and their ancillary facilities, such as broken insulators, corroded metal parts, and foreign object interference. These anomalies may lead to power grid failures or safety hazards; early detection and warnings will help prevent accidents.

[0004] Currently, power distribution network tower components are primarily identified using visible light images acquired by cameras, followed by target detection algorithms. However, these visible light image-based component identification algorithms can be sensitive to factors such as lighting conditions, angle, and occlusion, leading to misidentification or missed detections. Furthermore, power distribution network tower components exhibit significant diversity; identical components with different models and conditions may not be adequately identified using only visible light images. Summary of the Invention

[0005] This invention provides a method, apparatus, and drone for unmanned aerial vehicle (UAV) inspection of pole components in power distribution networks, in order to solve the problem of low accuracy in current identification models.

[0006] In a first aspect, embodiments of the present invention provide a method for unmanned aerial vehicle (UAV) inspection of pole components in a power distribution network, comprising:

[0007] The system acquires a real-time image set and determines the identification information of the detected objects based on the image set; wherein the image set includes visible light images and multispectral images of the detected objects;

[0008] The first position of the UAV is determined based on the position and size of the detected object in the visible light image and the standard image; wherein, the standard image is determined from a standard image library based on the identification information of the detected object;

[0009] Acquire first environmental data and first multispectral image; the first multispectral image is taken by the UAV at the first location, and the first environmental data is the data of the environment in which the UAV was located when it took the first multispectral image;

[0010] Based on the first environmental data and the environmental data when the standard multispectral image was captured, the spectral reflectance of the detected object in the first multispectral image was corrected to obtain the first spectral reflectance curve of the detected object;

[0011] The analyte is detected based on the first spectral reflectance curve and the standard spectral reflectance curve.

[0012] In one possible implementation, determining the identification information of the detected objects on the image set based on the image set includes:

[0013] The multispectral image is input into the multispectral image detection module to obtain the category of the detected object;

[0014] Based on the category of the detected object, the visible light image, and the category boundary detection box template set, the boundary detection box of the detected object on the visible light image is determined.

[0015] In one possible implementation, determining the first position of the UAV based on the position and size of the detected object in a visible light image and a standard image includes:

[0016] Based on the category of the detected object, the boundary detection box template corresponding to the detected object is selected from the category boundary detection box template set;

[0017] Based on the boundary detection box template corresponding to the detected object, the boundary detection box of the detected object in the standard image containing the detected object is determined.

[0018] The first position of the UAV is determined based on the positions of the boundary detection boxes in the visible light image and the standard image, as well as the size of the detected object in the boundary detection box in the visible light image and the size of the detected object in the boundary detection box in the standard image.

[0019] In one possible implementation, the first position of the UAV is determined based on the positions of the boundary detection boxes in the visible light image and the boundary detection boxes in the standard image, as well as the sizes of the detected object within the boundary detection boxes in the visible light image and the standard image, respectively. This includes:

[0020] Based on the size of the detected object in the boundary detection box of the visible light image and the size of the boundary detection box in the standard image, the first distance that the UAV needs to adjust is determined; wherein, the first distance is the vertical distance between the UAV and the center point of the detected object, the size of the detected object in the boundary detection box of the second visible light image is equal to the size of the boundary detection box in the standard image, and the second visible light image is obtained by the UAV after adjusting the first distance;

[0021] Based on the positions of the boundary detection boxes in the visible light image and the boundary detection boxes in the standard image, a second distance that the UAV needs to adjust is determined; wherein, the second distance is the distance from the center point of the imaging device on the UAV projected onto the surface of the object to the center point of the object; the position of the boundary detection box in the third visible light image is the same as the position of the boundary detection box in the standard image; the third visible light image and the standard image have the same size; the third visible light image is obtained by the UAV after adjusting the second distance.

[0022] The first position of the drone is determined based on the first distance and the second distance.

[0023] In one possible implementation, the environmental data includes temperature and illumination, the first environmental data includes a first temperature and a first illumination, and the standard environmental data includes a standard temperature and a standard illumination.

[0024] Based on the first environmental data and the standard environmental data at the time of capturing the standard multispectral image, the spectral reflectance of the detected object in the first multispectral image is corrected, including:

[0025] The first variation factor is determined based on the spectral variation curves of the first temperature, the standard temperature, and the preset temperature.

[0026] The second variation factor is determined based on the spectral variation curves of the first illumination, standard illumination, and preset illumination.

[0027] The spectral reflectance of the detected object in the first multispectral image is corrected based on the first and second variation factors.

[0028] In one possible implementation, the spectral reflectance of the detected object in the first multispectral image is corrected based on a first change factor and a second change factor, including:

[0029] The first and second variation factors are input into the reflectance correction model to obtain the third variation factor; the reflectance correction model is a neural network model, which is trained based on historical temperature changes and historical illumination changes.

[0030] The spectral reflectance of the detected object in the first multispectral image is corrected based on the third variation factor.

[0031] In one possible implementation, the standard multispectral image and the standard image are images containing the detected object taken at the same location, and the standard spectral reflectance curve is the spectral reflectance curve of the detected object in the standard multispectral image;

[0032] The analyte is detected based on the first spectral reflectance curve and the standard spectral reflectance curve, including:

[0033] When the difference between the absolute value of the spectral reflectance of the target wavelength in the first spectral reflectance curve and the absolute value of the spectral reflectance in the standard spectral reflectance curve is greater than a preset difference threshold, it is determined that the detected substance is abnormal; where the target wavelength is any wavelength in the spectral reflectance curve.

[0034] When the sum of the absolute values ​​of the spectral reflectance of all target wavelengths in the first spectral reflectance curve and the absolute values ​​of the spectral reflectance in the standard spectral reflectance curve is greater than a first preset sum threshold or less than a second preset sum threshold, it is determined that the detected substance is abnormal; wherein, the first preset sum threshold is greater than the second preset sum threshold.

[0035] In one possible implementation, the method also includes:

[0036] When an abnormality is detected, the first multispectral image, the standard multispectral image, the first spectral reflectance curve, and the standard spectral reflectance curve are transmitted back to the central server.

[0037] Secondly, embodiments of the present invention provide a drone inspection device for pole and tower components in a power distribution network, comprising:

[0038] The first acquisition module is used to acquire a set of images acquired in real time and determine the identification information of the detected objects on the image set based on the image set; wherein, the image set includes visible light images and multispectral images of the detected objects;

[0039] The determination module is used to determine the first position of the UAV based on the position and size of the detected object in the visible light image and the standard image; wherein, the standard image is determined from a standard image library based on the identification information of the detected object;

[0040] The second acquisition module is used to acquire first environmental data and a first multispectral image; the first multispectral image is taken by the UAV at the first position, and the first environmental data is the data of the environment in which the UAV was located when it took the first multispectral image;

[0041] The processing module is used to correct the spectral reflectance of the detected object in the first multispectral image based on the first environmental data and the environmental data when the standard multispectral image was captured, so as to obtain the first spectral reflectance curve of the detected object;

[0042] The detection module is used to detect the analyte based on the first spectral reflectance curve and the standard spectral reflectance curve.

[0043] Thirdly, embodiments of the present invention provide a drone, including a drone inspection device for pole components of a power distribution network based on knowledge distillation and lightweight processing. The drone inspection device for pole components of a power distribution network is deployed in the drone and is used to perform inspections based on any one of the methods in the first aspect.

[0044] This invention provides a method, apparatus, and drone for unmanned aerial vehicle (UAV) inspection of pole components in a power distribution network. First, by acquiring a set of images collected in real time by the UAV, and determining the identification information of the objects to be inspected in the image set based on the image set, the category of the object is determined, facilitating further inspection. Next, to improve inspection accuracy, the UAV's inspection position needs to be determined. This can be achieved by determining the position and size of the object in the visible light image and a standard image, ensuring that the multispectral image captured by the UAV at the first position matches the position in the standard multispectral image. Then, first environmental data and a first multispectral image are acquired at the first position. Since the environment significantly affects the spectrum, the spectral reflectance of the object in the first multispectral image needs to be corrected based on the first environmental data and the environmental data at the time the standard multispectral image was captured, resulting in a first spectral reflectance curve for the object. Finally, the object is inspected based on the first spectral reflectance curve and the standard spectral reflectance curve. This not only improves the accuracy of UAV inspection but also increases inspection efficiency. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating the implementation of the UAV inspection method for power distribution network pole components provided in this embodiment of the invention.

[0047] Figure 2 This is a schematic diagram of the structure of the twin model provided in the embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of the UAV inspection device for pole components in power distribution networks provided in an embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram of the structure of the UAV provided in an embodiment of the present invention. Detailed Implementation

[0050] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0051] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0052] Figure 1 The implementation flowchart of the UAV inspection method for power distribution network pole components provided in this embodiment of the invention is described in detail below:

[0053] S110. Acquire the real-time image set and determine the identification information of the detected objects in the image set based on the image set.

[0054] The drone is equipped with a multispectral camera, which can acquire visible light and multispectral images of the object being detected in real time.

[0055] In addition, the drone is equipped with an inspection device that uses knowledge distillation and lightweight processing. This device can reduce the number of parameters for component identification and anomaly detection through knowledge distillation. It can also process the images collected in real time during flight and judge whether the state of the detected object is abnormal in real time. This not only saves computing resources and improves inference speed, but also meets the performance requirements of edge modules.

[0056] In some embodiments, lightweight multispectral image detection modules and processing devices can be deployed on drones using hardware and software platforms such as NVIDIA Jetson and TensorRT. The acquired images or videos are then fed into DeepStream, reducing performance losses associated with video decoding, encoding, and streaming transmission. The video is then converted into images.

[0057] In this embodiment, the original component identification and anomaly detection methods can first be optimized and simplified through knowledge distillation, reducing the number of model parameters. Knowledge distillation technology transfers knowledge between the auxiliary model and the target model, allowing the target model to learn information such as the feature representations and category distributions of the auxiliary model, thereby improving the model's generalization ability and inference speed.

[0058] Then, the knowledge distillation method is quantized and pruned. Specifically, first, the parameters are converted from high-precision floating-point numbers to low-precision floating-point numbers. Then, redundant neurons are identified and removed through pruning. The combination of quantization and pruning effectively reduces the model size, computational cost, and memory footprint, improving inference speed and making it more adaptable to the resource constraints of edge devices.

[0059] Finally, by connecting video streams of different spectra through DeepStream, rapid video stream decoding, encoding, and transmission are achieved, along with efficient streaming media processing capabilities. Real-time analysis of streaming media via DeepStream effectively reduces performance loss, ensuring smooth and efficient data processing, and providing reliable video data support for subsequent component identification and anomaly detection. Furthermore, it can efficiently and quickly convert video into images.

[0060] In some embodiments, the UAV is equipped with a multispectral image detection module that employs knowledge distillation and lightweight processing. After acquiring a multispectral image of the object to be detected, the multispectral image can be input into the multispectral image detection module to obtain the category of the object.

[0061] The objects to be tested can be tower components such as the tower body, conductors, insulators, and grounding wires of the power distribution tower.

[0062] In this embodiment, the multispectral image detection module includes a backbone network unit, a multispectral channel attention mechanism unit, and a detection and localization unit. The backbone network unit acquires features from different spectra and performs feature fusion based on channel dimensions in the neck network. By introducing the channel attention mechanism unit, the importance of each channel in the acquired feature map is determined, and each feature is assigned a corresponding weight according to its importance, thereby allowing the network to focus on certain feature channels. This enhances the channels of the feature map that are useful for the part recognition task and suppresses feature channels that are not useful for the current task. Finally, the acquired features are detected and localized by the detection and localization unit.

[0063] Specifically, the backbone network units can use the Darknet53 network, and the feature extraction steps are as follows:

[0064] First, before feature extraction, CSPDarknet53 performs neighbor downsampling to generate four complementary but distinct sub-images, converting width and height information into channel-dimensional information. After concatenating the four images, a new 12-channel image is obtained, which is then subjected to a conv operation to obtain a double-downsampled feature map without information loss. Assuming the image shape is Nx3xHxW, slicing transforms it into a Nx12xH / 2xW / 2 feature map, followed by a conv operation to obtain an Nx32xH / 2xW / 2 feature map.

[0065] Next, the C3 module is a key component of CSPDarknet53, containing three standard convolutional layers and multiple BottleNeck modules. The BottleNeck modules borrow from the ResNet residual structure, where the left branch halves the number of channels in the feature map using a Conv1x1 convolution, then doubles the number of channels by extracting features using a Conv3x3 convolution. The right branch uses a shortcut to perform a residual connection, adding it to the output features of the left branch. Ultimately, this effectively solves the gradient problem while extracting features at three different scales.

[0066] Specifically, the processing steps of the multispectral channel attention mechanism unit are as follows:

[0067] First, Concat multispectral feature fusion.

[0068] In the neck network, feature maps from different spectra are concatenated along the channel dimension to achieve multispectral feature fusion. By concatenating feature maps, information from different spectra can be integrated, thus fully considering information such as the outline, material, and temperature of the tower components.

[0069] Secondly, the multispectral channel attention mechanism.

[0070] The feature map processing for multispectral stitching is based on the Channel Attention Mechanism (CAM), which consists of two parts: compression and activation. The compression part aims to compress information in the global space, then performs feature learning along the channel dimension to determine the importance of each channel, and finally assigns different weights to each channel through the activation part.

[0071] The compression part uses a squeeze operation to compress the feature map into a feature vector through global average pooling. The input feature map has dimensions H×W×C, representing height, width, and number of channels, respectively. Compression reduces the feature map dimension from H×W×C to 1×1×C.

[0072] In the excitation part, the excitation operation uses a fully connected layer to reduce the number of channels in the feature map to 1 / r of the original number. Then, after passing through the Swish function and another fully connected layer, the channel dimension of the feature map is increased to 1×1×C. Finally, it is transformed into a normalized weight vector between 0 and 1 using the sigmoid function. Through the scale operation, the normalized weights are multiplied channel by channel with the original input feature map to generate a weighted feature map.

[0073] The processing steps for detecting and positioning units are as follows:

[0074] The main part of the detection head consists of three detectors, whose network structure comprises only three Conv1x1 layers, corresponding to three detection feature layers. It utilizes grid-based anchors on feature maps at different scales for target detection. The main process is as follows: after obtaining the output, it is compared with the real data annotations, the localization loss, classification loss, and confidence loss are calculated, and then the data format is reshaped as needed, while the original grid coordinates are activated and localized accordingly.

[0075] In some embodiments, in order to accurately locate the position of the multispectral camera on the UAV relative to the object being detected, it is also necessary to determine the boundary detection box of the object on the visible light image based on the object's category, the visible light image, and the category boundary detection box template set. At this point, the boundary detection box of the object can be displayed on the visible light image.

[0076] The category boundary detection box template set includes a boundary detection box template corresponding to each type of object to be detected, and the boundary detection box template can completely place the object to be detected within the box.

[0077] The visible light images and multispectral images captured by the multispectral camera in this invention are of the same size, and the visible light images and multispectral images acquired in this invention are for a single detection object.

[0078] S120. Determine the first position of the UAV based on the position and size of the detected object in the visible light image and the standard image.

[0079] The standard images are determined from a standard image library based on the identification information of the detected objects. The standard image library stores standard images of all detected objects. These standard images are captured under specific shooting locations and environmental conditions. It should be noted that each detected object corresponds to one standard image, and these standard images are pre-captured. The environmental conditions here refer to temperature and illumination. The standard images and visible light images are also the same size. Each detected object corresponds to only one standard image, and each standard image contains only one detected object. The initial position of the UAV is also determined based on the position and size of the detected object in both the visible light image and the standard image.

[0080] When capturing visible light or multispectral images, different shooting distances and shooting angles may cause the captured images to be shifted, stretched, scaled, or transformed. This can significantly affect the subsequent inspection results and reduce the accuracy of the inspection. In order to improve the accuracy of the inspection results, it is necessary to determine the shooting position of the drone.

[0081] In some embodiments, after determining the category of the object to be detected, the boundary detection box template corresponding to the object can be selected from the category boundary detection box template set based on the category of the object to be detected.

[0082] Then, based on the boundary detection box template corresponding to the detected object, the boundary detection box of the detected object in the standard image containing the detected object is determined.

[0083] Finally, based on the positions of the boundary detection boxes in the visible light image and the standard image, as well as the size of the detected object in the boundary detection box in the visible light image and the size of the detected object in the boundary detection box in the standard image, the first position of the UAV is determined.

[0084] In this embodiment, the vertical distance between the drone and the object can be adjusted by modifying the size of the object's bounding box in the visible light image and its bounding box in the standard image. This ensures that the newly captured image and the standard image are taken in the same vertical plane relative to the object. By adjusting the positions of the bounding boxes of the object in the visible light image and the standard image, the shooting position of the multispectral camera on the drone can be made the same as the shooting position of the standard image.

[0085] Specifically, based on the size of the detected object within the boundary detection box of the visible light image and its size within the boundary detection box of the standard image, the first distance that the UAV needs to adjust is determined. Here, the first distance is the vertical distance between the UAV and the center point of the detected object.

[0086] When the vertical distance between the drone and the center point of the object being detected varies, the size of the object captured in the image will differ, resulting in different sizes within the boundary detection box. To ensure a fixed drone shooting position, a first distance that the drone needs to adjust can be determined first. After adjusting the drone based on the first distance, a second visible light image is captured. The size of the object within the boundary detection box in the second visible light image is equal to the size within the boundary detection box in the standard image.

[0087] Based on the positions of the boundary detection boxes in the visible light image and the standard image, a second distance that the UAV needs to adjust is determined. This second distance is the horizontal and vertical distance from the center point of the UAV's imaging device projected onto the surface of the object being detected, relative to the center point of the object. After adjusting based on the second distance, the UAV captures a third visible light image. The positions of the boundary detection boxes in the third visible light image are the same as those in the standard image.

[0088] Finally, based on the first distance and the second distance, the first position of the drone is determined.

[0089] S130, Acquire first environmental data and first multispectral image.

[0090] The first multispectral image was taken by the drone in its first position. Once the drone is in the first position, the gimbal and focus are automatically adjusted. During the adjustment, the exposure is automatically determined based on the light intensity at the first position to ensure that the exposure of each shot remains within a certain threshold range.

[0091] The first environmental data is the data of the environment in which the drone was located when it took the first multispectral image.

[0092] In some embodiments, the environmental data here includes temperature and illumination. Besides the shooting distance, temperature and illumination at the time of shooting also affect the multispectral image. Therefore, to improve the accuracy of inspections, the effects of temperature and illumination need to be considered.

[0093] S140. Based on the first environmental data and the standard environmental data when the standard multispectral image was taken, the spectral reflectance of the detected object in the first multispectral image is corrected to obtain the first spectral reflectance curve of the detected object.

[0094] In some embodiments, environmental data includes temperature and light intensity, first environmental data includes a first temperature and a first light intensity, and standard environmental data includes a standard temperature and a standard light intensity.

[0095] In this embodiment, a first variation factor can be determined firstly based on the spectral variation curves of a first temperature, a standard temperature, and a preset temperature. Then, a second variation factor can be determined based on the spectral variation curves of a first illumination, a standard illumination, and a preset illumination. Finally, the spectral reflectance of the detected object in the first multispectral image is corrected based on the first and second variation factors.

[0096] Specifically, the preset temperature spectral variation curve is determined based on historical temperatures and the corresponding spectral reflectance. The preset illumination spectral variation curve is determined based on historical illumination and the corresponding spectral reflectance.

[0097] Since temperature and illumination both affect spectral reflectance, adjusting spectral reflectance based on only a single factor is inaccurate. To further improve the accuracy of the spectral reflectance of the detected object in the first multispectral image, a reflectance correction model can be constructed. This reflectance correction model is stored in the UAV through distillation and lightweighting processes.

[0098] The reflectance correction model is a neural network model trained on historical temperature and illumination variations. The model takes temperature and illumination variation factors as inputs. Since spectral reflectance is not solely influenced by temperature or illumination, but rather by both, the output third variation factor is also influenced by both. Therefore, the reflectance correction model is trained using historical temperature and illumination variation factors, along with the current spectral reflectance based on the given temperature and illumination, to accurately predict this third variation factor resulting from the combined effects of temperature and illumination.

[0099] Specifically, the first and second variation factors can be input into the reflectance correction model to obtain the third variation factor. Then, the spectral reflectance of the detected object in the first multispectral image is corrected based on the third variation factor.

[0100] S150. The analyte is detected based on the first spectral reflectance curve and the standard spectral reflectance curve.

[0101] In some embodiments, detection can be based on reflectance curves. The standard multispectral image and the standard image are images containing the object to be detected, taken at the same location, and the standard spectral reflectance curve is the spectral reflectance curve of the object to be detected in the standard multispectral image.

[0102] In this embodiment, when the difference between the absolute value of the spectral reflectance of the target wavelength in the first spectral reflectance curve and the absolute value of the spectral reflectance in the standard spectral reflectance curve is greater than a preset difference threshold, it is determined that the detected object is abnormal. The target wavelength can be any wavelength in the spectral reflectance curve. The preset difference threshold is a fixed value determined based on the actual application scenario and the detected object.

[0103] Furthermore, if the sum of the absolute values ​​of the spectral reflectance of all target wavelengths in the first spectral reflectance curve and the absolute values ​​of the spectral reflectance in the standard spectral reflectance curve is greater than a first preset sum threshold, or less than a second preset sum threshold, then an anomaly in the detected object can also be determined. The first preset sum threshold is greater than the second preset sum threshold, and both the first and second preset sum thresholds are fixed values ​​determined based on the actual application scenario and the detected object.

[0104] The specific location of the anomaly can also be determined based on the difference between the first spectral reflectance curve and the standard spectral reflectance curve for all wavelengths.

[0105] Furthermore, in some embodiments, anomaly detection of components can also be achieved using Siamese networks when detecting objects. A Siamese network is a special network structure capable of learning the similarity or difference between two inputs. First, historical images and the current component image are input into two branches of the Siamese network to extract features. Then, by calculating the similarity between the features output by the two branches, if the similarity is below a set threshold, the current component can be determined to be an anomaly.

[0106] In this embodiment, such as Figure 2 As shown, the two branches of the Siamese network process historical normal images and current part images, respectively. The two convolutional networks on the left and right sides have identical structures and share the same weights W. The input data is a set of images (X1, X2, Y), where X1 represents a historical image, X2 represents the current image, Y = 0 indicates that part X2 is normal, and Y = 1 indicates that part X2 is abnormal. Therefore, normal pairs are (X1, X2+, 0), and abnormal pairs are (X1, X2-, 1). For two different inputs X1 and X2, high-level feature representations are output as Gw(X1) and Gw(X2), respectively. These feature representations capture key information in the image, such as edges, texture, and shape, providing useful features for subsequent anomaly detection.

[0107] For a historical normal image X1, if another normal image X2+ of the same part is given, the Euclidean distance between the model's output features Gw(X1) and Gw(X2+) is small, and the loss value becomes large. If another abnormal image X2- of the same part is given, the distance between the model's output features Gw(X1) and Gw(X2-) is relatively large. Therefore, D(X1, X2+) <D(X1,X2-)。

[0108] Anomaly detection for components is performed based on feature similarity output from the Siamese network. If the similarity value between the current component image and historical component images is lower than a set threshold k, the system determines the current component as abnormal and triggers a corresponding anomaly alarm mechanism. The advantage of this method is that it de-emphasizes labels, giving the network good scalability and allowing it to classify categories that have not been trained on.

[0109] In some embodiments, the detector can be detected by combining the spectral reflectance curve and the twin model, which can further improve the accuracy of detection.

[0110] In some embodiments, when an anomaly is detected, the first multispectral image, the standard multispectral image, the first spectral reflectance curve, and the standard spectral reflectance curve can be compressed and transmitted back to the central server. The central server restores the compressed image using an image restoration service, and further, based on the comparison of the first spectral reflectance curve and the standard spectral reflectance curve, can determine the specific location of the anomaly, helping to understand the status and changes of the power distribution tower components and to take necessary measures in a timely manner.

[0111] In this embodiment, artificial neural networks can be used for predictive coding, transform coding, vector quantization, etc., based on neural network technology, combined with lossy and lossless compression algorithms, to preserve as much complete image information as possible and seek the possibility of a large compression ratio.

[0112] Furthermore, deep neural networks have a large number of layers and parameters, placing extremely high demands on memory and computing power when performing tasks. Deep neural networks cannot meet the needs of drones in terms of memory, energy consumption, and bandwidth. Therefore, it is necessary to use lightweight model compression techniques, combining pruning, weight sharing, and quantization, to reduce the number of model parameters and size, making them suitable for conventional edge devices.

[0113] After image compression, image decoding is also required. Image compression technology reduces data transmission and storage overhead. Compressed image data can be transmitted back to the central server more quickly and efficiently for further analysis and recording, helping to understand the status and changes of distribution network tower components in a timely manner.

[0114] The UAV inspection method, apparatus, and UAV provided by this invention first acquire a set of images collected in real time by the UAV, and determine the identification information of the objects to be detected in the image set based on the image set, thereby determining the category of the objects to be detected, facilitating further detection of the objects. Next, to improve detection accuracy, it is also necessary to determine the detection position of the UAV. This can be done by determining the position and size of the objects to be detected in visible light images and standard images, ensuring that the position of the multispectral image captured by the UAV at the first position is the same as that of the standard multispectral image. Then, first environmental data and a first multispectral image are acquired at the first position. Since the environment has a significant impact on the spectrum, it is also necessary to correct the spectral reflectance of the objects to be detected in the first multispectral image based on the first environmental data and the environmental data at the time the standard multispectral image was captured, obtaining a first spectral reflectance curve of the objects to be detected. Finally, the objects to be detected are detected based on the first spectral reflectance curve and the standard spectral reflectance curve. Therefore, this not only improves the detection accuracy of UAV inspection but also increases detection efficiency.

[0115] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0116] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0117] Figure 3 A schematic diagram of a drone inspection device 300 for power distribution network pole components provided in an embodiment of the present invention is shown. For ease of explanation, only the parts relevant to the embodiment of the present invention are shown, and are described in detail below:

[0118] like Figure 3 As shown, a drone inspection device 300 for pole components in a power distribution network includes:

[0119] The first acquisition module 310 is used to acquire a set of images acquired in real time and determine the identification information of the detected object on the image set based on the image set; wherein, the image set includes visible light images and multispectral images of the detected object;

[0120] The determination module 320 is used to determine the first position of the UAV based on the position and size of the detected object in the visible light image and the standard image; wherein the standard image is determined from a standard image library based on the identification information of the detected object;

[0121] The second acquisition module 330 is used to acquire first environmental data and a first multispectral image; the first multispectral image is taken by the UAV at the first position, and the first environmental data is the data of the environment in which the UAV was located when it took the first multispectral image;

[0122] Processing module 340 is used to correct the spectral reflectance of the detected object in the first multispectral image based on the first environmental data and the environmental data when the standard multispectral image was captured, so as to obtain the first spectral reflectance curve of the detected object;

[0123] The detection module 350 is used to detect the analyte based on the first spectral reflectance curve and the standard spectral reflectance curve.

[0124] In one possible implementation, the first acquisition module 310 is used to input the multispectral image into the multispectral image detection module to obtain the category of the detected object;

[0125] Based on the category of the detected object, the visible light image, and the category boundary detection box template set, the boundary detection box of the detected object on the visible light image is determined.

[0126] In one possible implementation, the determining module 320 is used to filter out the boundary detection box template corresponding to the detected object from the category boundary detection box template set based on the category of the detected object;

[0127] Based on the boundary detection box template corresponding to the detected object, the boundary detection box of the detected object in the standard image containing the detected object is determined.

[0128] The first position of the UAV is determined based on the positions of the boundary detection boxes in the visible light image and the standard image, as well as the size of the detected object in the boundary detection box in the visible light image and the size of the detected object in the boundary detection box in the standard image.

[0129] In one possible implementation, the determining module 320 is used to determine a first distance that the UAV needs to adjust based on the size of the detected object in the boundary detection box of the visible light image and the size of the detected object in the boundary detection box of the standard image; wherein, the first distance is the vertical distance between the UAV and the center point of the detected object, the size of the detected object in the boundary detection box of the second visible light image is equal to the size of the detected object in the boundary detection box of the standard image, and the second visible light image is obtained by the UAV after adjusting the first distance;

[0130] Based on the positions of the boundary detection boxes in the visible light image and the boundary detection boxes in the standard image, a second distance that the UAV needs to adjust is determined; wherein, the second distance is the distance from the center point of the imaging device on the UAV projected onto the surface of the object to the center point of the object; the position of the boundary detection box in the third visible light image is the same as the position of the boundary detection box in the standard image; the third visible light image and the standard image have the same size; the third visible light image is obtained by the UAV after adjusting the second distance.

[0131] The first position of the drone is determined based on the first distance and the second distance.

[0132] In one possible implementation, the environmental data includes temperature and illumination, the first environmental data includes a first temperature and a first illumination, and the standard environmental data includes a standard temperature and a standard illumination.

[0133] Processing module 340 is used to determine a first change factor based on the spectral change curves of a first temperature, a standard temperature, and a preset temperature;

[0134] The second variation factor is determined based on the spectral variation curves of the first illumination, standard illumination, and preset illumination.

[0135] The spectral reflectance of the detected object in the first multispectral image is corrected based on the first and second variation factors.

[0136] In one possible implementation, the processing module 340 is used to input the first change factor and the second change factor into the reflectance correction model to obtain the third change factor; wherein, the reflectance correction model is a neural network model, which is trained based on historical temperature changes and historical illumination changes.

[0137] The spectral reflectance of the detected object in the first multispectral image is corrected based on the third variation factor.

[0138] In one possible implementation, the standard multispectral image and the standard image are images containing the detected object taken at the same location, and the standard spectral reflectance curve is the spectral reflectance curve of the detected object in the standard multispectral image;

[0139] The detection module 350 is used to determine that the detected object is abnormal when the difference between the absolute value of the spectral reflectance of the target wavelength in the first spectral reflectance curve and the absolute value of the spectral reflectance in the standard spectral reflectance curve is greater than a preset difference threshold; wherein, the target wavelength is any wavelength in the spectral reflectance curve;

[0140] When the sum of the absolute values ​​of the spectral reflectance of all target wavelengths in the first spectral reflectance curve and the absolute values ​​of the spectral reflectance in the standard spectral reflectance curve is greater than a first preset sum threshold or less than a second preset sum threshold, it is determined that the detected substance is abnormal; wherein, the first preset sum threshold is greater than the second preset sum threshold.

[0141] In one possible implementation, the detection module 350 is used to send back the first multispectral image, the standard multispectral image, the first spectral reflectance curve, and the standard spectral reflectance curve to the central server when an abnormality is detected.

[0142] Figure 4 The invention illustrates a drone provided by an embodiment of the present invention, including a drone inspection device 300 for pole components of a power distribution network based on knowledge distillation and lightweight processing. The drone inspection device 300 for pole components of a power distribution network is deployed in the drone and is used to perform inspections based on any of the methods in the first aspect.

[0143] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0144] Those skilled in the art will recognize that the templates, units, and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0145] If the module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the above embodiments of the UAV inspection method for power distribution network tower components. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0146] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for unmanned aerial vehicle (UAV) inspection of pole and tower components in a power distribution network, characterized in that, include: The system acquires a set of images collected in real time and inputs the multispectral images into the multispectral image detection module to obtain the category of the detected object. Based on the category of the detected object, the visible light image, and the category boundary detection box template set, the boundary detection box of the detected object on the visible light image is determined; wherein, the image set includes the visible light image and multispectral image of the detected object; the detected object is the tower body, conductor, insulator, or grounding wire of the power distribution tower; Based on the category of the detected object, the boundary detection box template corresponding to the detected object is selected from the category boundary detection box template set; Based on the boundary detection box template corresponding to the detected object, the boundary detection box of the detected object in the standard image containing the detected object is determined. Based on the size of the detected object in the boundary detection box of the visible light image and the size of the detected object in the boundary detection box of the standard image, a first distance that the UAV needs to adjust is determined; wherein, the first distance is the vertical distance between the UAV and the center point of the detected object, the size of the detected object in the boundary detection box of the second visible light image is equal to the size of the detected object in the boundary detection box of the standard image, and the second visible light image is obtained by the UAV after adjusting the first distance; Based on the positions of the boundary detection boxes on the visible light image and the boundary detection boxes in the standard image, a second distance that the UAV needs to adjust is determined; wherein, the second distance is the distance from the center point of the shooting device on the UAV projected onto the surface of the object to the center point of the object; the position of the boundary detection box on the third visible light image is the same as the position of the boundary detection box in the standard image; the third visible light image and the standard image have the same size; the third visible light image is obtained by the UAV after adjusting the second distance. Based on the first distance and the second distance, the first position of the UAV is determined; wherein the standard image is determined from a standard image library based on the identification information of the detected object; Acquire first environmental data and first multispectral image; the first multispectral image is taken by the UAV at the first location, and the first environmental data is the data of the environment in which the UAV was located when it took the first multispectral image; Based on the first environmental data and the environmental data when the standard multispectral image was captured, the spectral reflectance of the detected object in the first multispectral image is corrected to obtain the first spectral reflectance curve of the detected object; The analyte is detected based on the first spectral reflectance curve and the standard spectral reflectance curve.

2. The UAV inspection method for pole and tower components in a power distribution network according to claim 1, characterized in that, The environmental data includes temperature and light intensity, the first environmental data includes a first temperature and a first light intensity, and the standard environmental data includes a standard temperature and a standard light intensity. Based on the first environmental data and the standard environmental data at the time of capturing the standard multispectral image, the spectral reflectance of the detected object in the first multispectral image is corrected, including: The first variation factor is determined based on the spectral variation curves of the first temperature, the standard temperature, and the preset temperature. The second variation factor is determined based on the spectral variation curves of the first illumination, standard illumination, and preset illumination. The spectral reflectance of the detected object in the first multispectral image is corrected based on the first change factor and the second change factor.

3. The UAV inspection method for pole and tower components in a power distribution network according to claim 2, characterized in that, The step of correcting the spectral reflectance of the detected object in the first multispectral image based on the first change factor and the second change factor includes: The first and second variation factors are input into the reflectance correction model to obtain the third variation factor; wherein, the reflectance correction model is a neural network model, which is trained based on historical temperature changes and historical illumination changes; The spectral reflectance of the detected object in the first multispectral image is corrected based on the third variation factor.

4. The UAV inspection method for pole and tower components in a power distribution network according to claim 1, characterized in that, The standard multispectral image and the standard image are images containing the detected object taken at the same location, and the standard spectral reflectance curve is the spectral reflectance curve of the detected object in the standard multispectral image; The detection of the analyte based on the first spectral reflectance curve and the standard spectral reflectance curve includes: When the difference between the absolute value of the spectral reflectance of the target wavelength in the first spectral reflectance curve and the absolute value of the spectral reflectance in the standard spectral reflectance curve is greater than a preset difference threshold, it is determined that the detected substance is abnormal; wherein, the target wavelength is any wavelength in the spectral reflectance curve; When the sum of the absolute values ​​of the spectral reflectance of all target wavelengths in the first spectral reflectance curve and the absolute values ​​of the spectral reflectance in the standard spectral reflectance curve is greater than a first preset sum threshold or less than a second preset sum threshold, it is determined that the detected substance is abnormal; wherein, the first preset sum threshold is greater than the second preset sum threshold.

5. The UAV inspection method for pole and tower components in a power distribution network according to claim 4, characterized in that, The method further includes: When an abnormality is detected, the first multispectral image, the standard multispectral image, the first spectral reflectance curve, and the standard spectral reflectance curve are transmitted back to the central server.

6. A drone inspection device for pole and tower components in a power distribution network, characterized in that, include: The first acquisition module is used to acquire the real-time image set and input the multispectral image into the multispectral image detection module to obtain the category of the detected object; Based on the category of the detected object, the visible light image, and the category boundary detection box template set, the boundary detection box of the detected object on the visible light image is determined; wherein, the image set includes the visible light image and multispectral image of the detected object; the detected object is the tower body, conductor, insulator, or grounding wire of the power distribution tower; The determination module is configured to: filter out a boundary detection box template corresponding to the detected object from the category boundary detection box template set based on the category of the detected object; determine the boundary detection box of the detected object in a standard image containing the detected object based on the boundary detection box template corresponding to the detected object; and determine a first distance that the UAV needs to adjust based on the size of the detected object in the boundary detection box of the visible light image and the size of the detected object in the boundary detection box of the standard image; wherein, the first distance is the vertical distance between the UAV and the center point of the detected object, the size of the detected object in the boundary detection box of the second visible light image is equal to the size of the boundary detection box of the standard image, and the second visible light image is the image of the UAV when adjusting the first distance. The following steps are taken: Based on the positions of the boundary detection boxes in the visible light image and the standard image, a second distance needs to be adjusted for the drone; wherein, the second distance is the distance from the center point of the drone's camera projected onto the surface of the object to the center point of the object; the positions of the boundary detection boxes in the third visible light image are the same as the positions of the boundary detection boxes in the standard image; the third visible light image and the standard image have the same size; the third visible light image is acquired by the drone after adjusting the second distance; based on the first distance and the second distance, a first position of the drone is determined; wherein, the standard image is determined from a standard image library based on the object's recognition information. The second acquisition module is used to acquire first environmental data and a first multispectral image; the first multispectral image is taken by the UAV at the first position, and the first environmental data is the data of the environment in which the UAV was located when it took the first multispectral image; The processing module is used to correct the spectral reflectance of the detected object in the first multispectral image based on the first environmental data and the environmental data when the standard multispectral image was captured, so as to obtain the first spectral reflectance curve of the detected object. The detection module is used to detect the analyte based on the first spectral reflectance curve and the standard spectral reflectance curve.

7. A drone, characterized in that, The invention includes a drone inspection device for pole components in a power distribution network, based on knowledge distillation and lightweight processing. The drone inspection device for pole components in a power distribution network is deployed in the drone and is used for inspection based on the method described in any one of claims 1-5.

Images